Hospital Medicine

A 58-year-old man with a remote history of endocarditis and no prior injection drug use was admitted to the inpatient medicine service with fever and concern for recurrent endocarditis. A transthoracic echocardiogram was unremarkable and the patient remained clinically stable. A transesophageal echocardiogram (TEE) was scheduled for the following morning, but during nursing rounds, the patient was missing from his room. Multiple staff members searched for the patient and eventually located him in the hospital lobby drinking a cup of coffee purchased from the cafeteria. Despite his opposition, he was escorted back to his room and advised to not leave the floor again. Later that day, the patient became frustrated and left the hospital before his scheduled TEE. He was subsequently lost to follow-up.

Patients are admitted to the hospital based upon a medical determination that the patient requires acute observation, evaluation, or treatment. Once admitted, healthcare providers may impose restrictions on the patient’s movement in the hospital, such as restrictions on leaving their assigned floor. Managing the movement of hospitalized patients poses significant challenges for the clinical staff because of the difficulty of providing a treatment environment that ensures safe and efficient delivery of care while promoting patients’ preferences for an unrestrictive environment that respects their independence.^1,2 Broad limits may make it easier for staff to care for patients and reduce concerns about liability, but they may also frustrate patients who may be medically, psychiatrically, and physically stable and do not require stringent monitoring (eg, completing a course of intravenous antibiotics or awaiting placement at outside facilities).

Although this issue has broad implications for patient safety and hospital liability, authoritative guidance and evidence-based literature are lacking. Without clear guidelines, healthcare staff members are likely to spend more time in managing each individual request to leave the floor because they do not have a systematic strategy for making fair and consistent decisions. Here, we describe the patient and institutional considerations when managing patient movement in the hospital. We refer to “patient movement” specifically as a patient’s choice to move to different locations within the hospital, but outside of their assigned room and/or floor. This does not include scheduled, supervised ambulation activities, such as physical therapy.

Practices that promote patient movement offer significant benefits and risks. Enhancing movement is likely to reduce the “physiologic disruption”³ of hospitalization while improving patients’ overall satisfaction and alignment with patient-centered care. Liberalized movement also promotes independence and ambulation that reduces the rate of physical deconditioning.⁴

Despite theoretical benefits, hospitals may be more concerned about adverse events related to patient movement, such as falls, the use of illicit substances, or elopement. Given that hospitals may be legally⁵ and financially responsible⁶ for adverse events associated with patient movement, allowances for off-floor movement should be carefully considered with input from risk management, physicians, nursing leadership, patient advocates, and hospital administration.

Additionally, unannounced movement off the floor may interfere with timely and efficient care by causing lapses in monitoring, such as cardiac telemetry,⁷ medication administration, and scheduled diagnostic tests. In these situations, the risks of patient absence from the floor are significant and may ultimately negate the benefits of continued hospitalization by compromising the central elements of patient care.

Patients’ requests to leave the hospital floor should be evaluated systemically and transparently to promote fair, high-value care. First, a request for liberalized movement should prompt physicians that the patient may no longer require hospitalization and may be ready for the transition to outpatient care.⁸ If the patient still requires inpatient care, then the medical practitioner should make a clinical determination if the patient is medically stable enough to leave their hospital floor. The provider should first identify when the liberalization of movement would be universally inappropriate, such as in patients who are physically unable to ambulate without posing significant harm to themselves. This includes an accidental fall (usually while walking⁵), which is one of the most commonly reported adverse events in an inpatient setting.⁹ Additionally, patients with significant cognitive impairments or those lacking in decision-making capacity may be restricted from leaving their floors unescorted, as they are at a higher risk of disorientation, falls, and death.¹⁰

In determining movement restrictions for patients in isolation, hospitals should refer to the existing guidelines on isolation precautions for the transmission of communicable infections^11,12 and neutropenic precautions.¹³ Additionally, movement restriction for patients who are isolated after screening positive for certain drug-resistant organisms (eg, methicillin-resistant Staphylococcus aureus and vancomycin-resistant enterococci) is controversial and should be evaluated based on the available medical evidence and standards.^14-16

When making a risk-benefit determination about movement, providers should also assess the intent and the potentially unmet needs behind the patient’s request. Patient-centered reasons for enhanced freedom of movement within the hospital include a desire for exercise, greater food choice, and visiting with loved ones, all of which can enable patients to manage the well-known inconveniences and stresses of hospitalization. In contrast, there may be concerns for other intentions behind leaving assigned floors based on the patient’s clinical history, such as the surreptitious use of illicit substances or attempts to elope from the hospital. Advising restriction of movement is justifiable if there is a significant concern for behavior that undermines the safe delivery of care. In patients with active substance use disorders, the appropriate treatment of pain or withdrawal symptoms may better address the patients’ unmet needs, but a lower threshold to restrict movement may be reasonable given the significant risks involved. However, given the widespread stigmatization of patients with substance use disorders,¹⁷ institutional policy and clinicians should adhere to systematic, transparent, and consistent risk assessments for all patients in order to minimize the potential for introducing or exacerbating disparities in care.

In order to work productively with admitted patients, strong practices honor patients’ autonomy by specifying when and how patients are informed of the institution’s expectations about and limitations to inpatient movement. For example, emergency room patients were less likely to elope when treatment expectations were established at the time of presentation by giving them information about wait times and triage protocol.¹⁸ Similarly, by preemptively discussing reasonable restrictions on movement as a part of informed consent for inpatient admission, physicians can establish patients’ expectations early in the admission process and foster a therapeutic alliance on the basis of the shared goals of safe and timely care.

Patients may request or even demand to leave the floor after a healthcare provider has determined that doing so would be unsafe and/or undermine the timely and efficient delivery of care. In these cases, shared decision-making (SDM) can help identify acceptable solutions within the identified constraints. SDM combines the physicians’ experience, expertise, and knowledge of medical evidence with patients’ values, needs, and preferences for care.¹⁹ If patients continue to request to leave the floor after the restriction has been communicated, physicians should discuss whether the current treatment plan should be renegotiated to include a relatively minor modification (eg, a change in the timing or route of administration of medication). If inpatient care cannot be provided safely within the patient’s preferences for movement and attempts to accommodate the patient’s preferences are unsuccessful, then a shift to discharge planning may be appropriate. A summary of this decision process is outlined in the Figure.

In mitigating the burdens of restriction on movement, hospitals may implement a range of options that address patients’ preferences while maintaining safety. Given the potential consequences of liberalized patient movement, it may be prudent to implement these safeguards as a compromise that addresses both the patients’ needs and the hospital’s concerns. These could include an escort for off-floor supervision, timed passes to leave the floor, or volunteers purchasing food for patients from the cafeteria. Creating open, supervised spaces within the hospital (eg, lounges) may also help provide the respite patients need, but in a safe and medically structured environment.

Returning to the introductory case example, we now present an alternative outcome in the context of the practices described above. On the morning of the scheduled TEE, a nurse noted that the patient was missing from his room. Before the staff began searching for the patient, they consulted the medical record which included the admission discussion and agreement to expectations for inpatient movement. The record also included an informed consent discussion indicating the minimal risks of leaving the floor, as the patient could ambulate independently and had no need for continuous monitoring. Finally, a physician’s order authorized the patient to be off the floor until 10 am. The patient returned to his room at 9:45 am and underwent a normal TEE, after which he was discharged home with outpatient follow-up.

The above scenario highlights the benefits of a comprehensive framework for patient movement practices that are transparent, fair, and systematic. Explicitly recognizing competing institutional and patient perspectives can prevent conflict and promote high-quality, safe, efficient, patient-centered care that only restricts the patient’s movement under specified and justifiable conditions. In developing strong hospital practices, institutions should refer to the relevant clinical and ethical standards and draw upon their institutional resources in risk management, clinical staff, and patient advocates.

Acknowledgments

The authors thank Dr. Neil Shapiro and Dr. David Chuquin for their constructive reviews of prior versions of this manuscript.

Disclosures

The authors have no financial conflicts of interest to disclose.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the U.S. Department of Veterans Affairs, the US Government, or the VA National Center for Ethics in Health Care.

A 58-year-old man with a remote history of endocarditis and no prior injection drug use was admitted to the inpatient medicine service with fever and concern for recurrent endocarditis. A transthoracic echocardiogram was unremarkable and the patient remained clinically stable. A transesophageal echocardiogram (TEE) was scheduled for the following morning, but during nursing rounds, the patient was missing from his room. Multiple staff members searched for the patient and eventually located him in the hospital lobby drinking a cup of coffee purchased from the cafeteria. Despite his opposition, he was escorted back to his room and advised to not leave the floor again. Later that day, the patient became frustrated and left the hospital before his scheduled TEE. He was subsequently lost to follow-up.

Patients are admitted to the hospital based upon a medical determination that the patient requires acute observation, evaluation, or treatment. Once admitted, healthcare providers may impose restrictions on the patient’s movement in the hospital, such as restrictions on leaving their assigned floor. Managing the movement of hospitalized patients poses significant challenges for the clinical staff because of the difficulty of providing a treatment environment that ensures safe and efficient delivery of care while promoting patients’ preferences for an unrestrictive environment that respects their independence.^1,2 Broad limits may make it easier for staff to care for patients and reduce concerns about liability, but they may also frustrate patients who may be medically, psychiatrically, and physically stable and do not require stringent monitoring (eg, completing a course of intravenous antibiotics or awaiting placement at outside facilities).

Although this issue has broad implications for patient safety and hospital liability, authoritative guidance and evidence-based literature are lacking. Without clear guidelines, healthcare staff members are likely to spend more time in managing each individual request to leave the floor because they do not have a systematic strategy for making fair and consistent decisions. Here, we describe the patient and institutional considerations when managing patient movement in the hospital. We refer to “patient movement” specifically as a patient’s choice to move to different locations within the hospital, but outside of their assigned room and/or floor. This does not include scheduled, supervised ambulation activities, such as physical therapy.

Practices that promote patient movement offer significant benefits and risks. Enhancing movement is likely to reduce the “physiologic disruption”³ of hospitalization while improving patients’ overall satisfaction and alignment with patient-centered care. Liberalized movement also promotes independence and ambulation that reduces the rate of physical deconditioning.⁴

Despite theoretical benefits, hospitals may be more concerned about adverse events related to patient movement, such as falls, the use of illicit substances, or elopement. Given that hospitals may be legally⁵ and financially responsible⁶ for adverse events associated with patient movement, allowances for off-floor movement should be carefully considered with input from risk management, physicians, nursing leadership, patient advocates, and hospital administration.

Additionally, unannounced movement off the floor may interfere with timely and efficient care by causing lapses in monitoring, such as cardiac telemetry,⁷ medication administration, and scheduled diagnostic tests. In these situations, the risks of patient absence from the floor are significant and may ultimately negate the benefits of continued hospitalization by compromising the central elements of patient care.

Patients’ requests to leave the hospital floor should be evaluated systemically and transparently to promote fair, high-value care. First, a request for liberalized movement should prompt physicians that the patient may no longer require hospitalization and may be ready for the transition to outpatient care.⁸ If the patient still requires inpatient care, then the medical practitioner should make a clinical determination if the patient is medically stable enough to leave their hospital floor. The provider should first identify when the liberalization of movement would be universally inappropriate, such as in patients who are physically unable to ambulate without posing significant harm to themselves. This includes an accidental fall (usually while walking⁵), which is one of the most commonly reported adverse events in an inpatient setting.⁹ Additionally, patients with significant cognitive impairments or those lacking in decision-making capacity may be restricted from leaving their floors unescorted, as they are at a higher risk of disorientation, falls, and death.¹⁰

In determining movement restrictions for patients in isolation, hospitals should refer to the existing guidelines on isolation precautions for the transmission of communicable infections^11,12 and neutropenic precautions.¹³ Additionally, movement restriction for patients who are isolated after screening positive for certain drug-resistant organisms (eg, methicillin-resistant Staphylococcus aureus and vancomycin-resistant enterococci) is controversial and should be evaluated based on the available medical evidence and standards.^14-16

When making a risk-benefit determination about movement, providers should also assess the intent and the potentially unmet needs behind the patient’s request. Patient-centered reasons for enhanced freedom of movement within the hospital include a desire for exercise, greater food choice, and visiting with loved ones, all of which can enable patients to manage the well-known inconveniences and stresses of hospitalization. In contrast, there may be concerns for other intentions behind leaving assigned floors based on the patient’s clinical history, such as the surreptitious use of illicit substances or attempts to elope from the hospital. Advising restriction of movement is justifiable if there is a significant concern for behavior that undermines the safe delivery of care. In patients with active substance use disorders, the appropriate treatment of pain or withdrawal symptoms may better address the patients’ unmet needs, but a lower threshold to restrict movement may be reasonable given the significant risks involved. However, given the widespread stigmatization of patients with substance use disorders,¹⁷ institutional policy and clinicians should adhere to systematic, transparent, and consistent risk assessments for all patients in order to minimize the potential for introducing or exacerbating disparities in care.

In order to work productively with admitted patients, strong practices honor patients’ autonomy by specifying when and how patients are informed of the institution’s expectations about and limitations to inpatient movement. For example, emergency room patients were less likely to elope when treatment expectations were established at the time of presentation by giving them information about wait times and triage protocol.¹⁸ Similarly, by preemptively discussing reasonable restrictions on movement as a part of informed consent for inpatient admission, physicians can establish patients’ expectations early in the admission process and foster a therapeutic alliance on the basis of the shared goals of safe and timely care.

Patients may request or even demand to leave the floor after a healthcare provider has determined that doing so would be unsafe and/or undermine the timely and efficient delivery of care. In these cases, shared decision-making (SDM) can help identify acceptable solutions within the identified constraints. SDM combines the physicians’ experience, expertise, and knowledge of medical evidence with patients’ values, needs, and preferences for care.¹⁹ If patients continue to request to leave the floor after the restriction has been communicated, physicians should discuss whether the current treatment plan should be renegotiated to include a relatively minor modification (eg, a change in the timing or route of administration of medication). If inpatient care cannot be provided safely within the patient’s preferences for movement and attempts to accommodate the patient’s preferences are unsuccessful, then a shift to discharge planning may be appropriate. A summary of this decision process is outlined in the Figure.

In mitigating the burdens of restriction on movement, hospitals may implement a range of options that address patients’ preferences while maintaining safety. Given the potential consequences of liberalized patient movement, it may be prudent to implement these safeguards as a compromise that addresses both the patients’ needs and the hospital’s concerns. These could include an escort for off-floor supervision, timed passes to leave the floor, or volunteers purchasing food for patients from the cafeteria. Creating open, supervised spaces within the hospital (eg, lounges) may also help provide the respite patients need, but in a safe and medically structured environment.

Returning to the introductory case example, we now present an alternative outcome in the context of the practices described above. On the morning of the scheduled TEE, a nurse noted that the patient was missing from his room. Before the staff began searching for the patient, they consulted the medical record which included the admission discussion and agreement to expectations for inpatient movement. The record also included an informed consent discussion indicating the minimal risks of leaving the floor, as the patient could ambulate independently and had no need for continuous monitoring. Finally, a physician’s order authorized the patient to be off the floor until 10 am. The patient returned to his room at 9:45 am and underwent a normal TEE, after which he was discharged home with outpatient follow-up.

The above scenario highlights the benefits of a comprehensive framework for patient movement practices that are transparent, fair, and systematic. Explicitly recognizing competing institutional and patient perspectives can prevent conflict and promote high-quality, safe, efficient, patient-centered care that only restricts the patient’s movement under specified and justifiable conditions. In developing strong hospital practices, institutions should refer to the relevant clinical and ethical standards and draw upon their institutional resources in risk management, clinical staff, and patient advocates.

Acknowledgments

The authors thank Dr. Neil Shapiro and Dr. David Chuquin for their constructive reviews of prior versions of this manuscript.

Disclosures

The authors have no financial conflicts of interest to disclose.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the U.S. Department of Veterans Affairs, the US Government, or the VA National Center for Ethics in Health Care.

A 58-year-old man with a remote history of endocarditis and no prior injection drug use was admitted to the inpatient medicine service with fever and concern for recurrent endocarditis. A transthoracic echocardiogram was unremarkable and the patient remained clinically stable. A transesophageal echocardiogram (TEE) was scheduled for the following morning, but during nursing rounds, the patient was missing from his room. Multiple staff members searched for the patient and eventually located him in the hospital lobby drinking a cup of coffee purchased from the cafeteria. Despite his opposition, he was escorted back to his room and advised to not leave the floor again. Later that day, the patient became frustrated and left the hospital before his scheduled TEE. He was subsequently lost to follow-up.

Patients are admitted to the hospital based upon a medical determination that the patient requires acute observation, evaluation, or treatment. Once admitted, healthcare providers may impose restrictions on the patient’s movement in the hospital, such as restrictions on leaving their assigned floor. Managing the movement of hospitalized patients poses significant challenges for the clinical staff because of the difficulty of providing a treatment environment that ensures safe and efficient delivery of care while promoting patients’ preferences for an unrestrictive environment that respects their independence.^1,2 Broad limits may make it easier for staff to care for patients and reduce concerns about liability, but they may also frustrate patients who may be medically, psychiatrically, and physically stable and do not require stringent monitoring (eg, completing a course of intravenous antibiotics or awaiting placement at outside facilities).

Although this issue has broad implications for patient safety and hospital liability, authoritative guidance and evidence-based literature are lacking. Without clear guidelines, healthcare staff members are likely to spend more time in managing each individual request to leave the floor because they do not have a systematic strategy for making fair and consistent decisions. Here, we describe the patient and institutional considerations when managing patient movement in the hospital. We refer to “patient movement” specifically as a patient’s choice to move to different locations within the hospital, but outside of their assigned room and/or floor. This does not include scheduled, supervised ambulation activities, such as physical therapy.

Practices that promote patient movement offer significant benefits and risks. Enhancing movement is likely to reduce the “physiologic disruption”³ of hospitalization while improving patients’ overall satisfaction and alignment with patient-centered care. Liberalized movement also promotes independence and ambulation that reduces the rate of physical deconditioning.⁴

Despite theoretical benefits, hospitals may be more concerned about adverse events related to patient movement, such as falls, the use of illicit substances, or elopement. Given that hospitals may be legally⁵ and financially responsible⁶ for adverse events associated with patient movement, allowances for off-floor movement should be carefully considered with input from risk management, physicians, nursing leadership, patient advocates, and hospital administration.

Additionally, unannounced movement off the floor may interfere with timely and efficient care by causing lapses in monitoring, such as cardiac telemetry,⁷ medication administration, and scheduled diagnostic tests. In these situations, the risks of patient absence from the floor are significant and may ultimately negate the benefits of continued hospitalization by compromising the central elements of patient care.

Patients’ requests to leave the hospital floor should be evaluated systemically and transparently to promote fair, high-value care. First, a request for liberalized movement should prompt physicians that the patient may no longer require hospitalization and may be ready for the transition to outpatient care.⁸ If the patient still requires inpatient care, then the medical practitioner should make a clinical determination if the patient is medically stable enough to leave their hospital floor. The provider should first identify when the liberalization of movement would be universally inappropriate, such as in patients who are physically unable to ambulate without posing significant harm to themselves. This includes an accidental fall (usually while walking⁵), which is one of the most commonly reported adverse events in an inpatient setting.⁹ Additionally, patients with significant cognitive impairments or those lacking in decision-making capacity may be restricted from leaving their floors unescorted, as they are at a higher risk of disorientation, falls, and death.¹⁰

In determining movement restrictions for patients in isolation, hospitals should refer to the existing guidelines on isolation precautions for the transmission of communicable infections^11,12 and neutropenic precautions.¹³ Additionally, movement restriction for patients who are isolated after screening positive for certain drug-resistant organisms (eg, methicillin-resistant Staphylococcus aureus and vancomycin-resistant enterococci) is controversial and should be evaluated based on the available medical evidence and standards.^14-16

When making a risk-benefit determination about movement, providers should also assess the intent and the potentially unmet needs behind the patient’s request. Patient-centered reasons for enhanced freedom of movement within the hospital include a desire for exercise, greater food choice, and visiting with loved ones, all of which can enable patients to manage the well-known inconveniences and stresses of hospitalization. In contrast, there may be concerns for other intentions behind leaving assigned floors based on the patient’s clinical history, such as the surreptitious use of illicit substances or attempts to elope from the hospital. Advising restriction of movement is justifiable if there is a significant concern for behavior that undermines the safe delivery of care. In patients with active substance use disorders, the appropriate treatment of pain or withdrawal symptoms may better address the patients’ unmet needs, but a lower threshold to restrict movement may be reasonable given the significant risks involved. However, given the widespread stigmatization of patients with substance use disorders,¹⁷ institutional policy and clinicians should adhere to systematic, transparent, and consistent risk assessments for all patients in order to minimize the potential for introducing or exacerbating disparities in care.

In order to work productively with admitted patients, strong practices honor patients’ autonomy by specifying when and how patients are informed of the institution’s expectations about and limitations to inpatient movement. For example, emergency room patients were less likely to elope when treatment expectations were established at the time of presentation by giving them information about wait times and triage protocol.¹⁸ Similarly, by preemptively discussing reasonable restrictions on movement as a part of informed consent for inpatient admission, physicians can establish patients’ expectations early in the admission process and foster a therapeutic alliance on the basis of the shared goals of safe and timely care.

Patients may request or even demand to leave the floor after a healthcare provider has determined that doing so would be unsafe and/or undermine the timely and efficient delivery of care. In these cases, shared decision-making (SDM) can help identify acceptable solutions within the identified constraints. SDM combines the physicians’ experience, expertise, and knowledge of medical evidence with patients’ values, needs, and preferences for care.¹⁹ If patients continue to request to leave the floor after the restriction has been communicated, physicians should discuss whether the current treatment plan should be renegotiated to include a relatively minor modification (eg, a change in the timing or route of administration of medication). If inpatient care cannot be provided safely within the patient’s preferences for movement and attempts to accommodate the patient’s preferences are unsuccessful, then a shift to discharge planning may be appropriate. A summary of this decision process is outlined in the Figure.

In mitigating the burdens of restriction on movement, hospitals may implement a range of options that address patients’ preferences while maintaining safety. Given the potential consequences of liberalized patient movement, it may be prudent to implement these safeguards as a compromise that addresses both the patients’ needs and the hospital’s concerns. These could include an escort for off-floor supervision, timed passes to leave the floor, or volunteers purchasing food for patients from the cafeteria. Creating open, supervised spaces within the hospital (eg, lounges) may also help provide the respite patients need, but in a safe and medically structured environment.

Returning to the introductory case example, we now present an alternative outcome in the context of the practices described above. On the morning of the scheduled TEE, a nurse noted that the patient was missing from his room. Before the staff began searching for the patient, they consulted the medical record which included the admission discussion and agreement to expectations for inpatient movement. The record also included an informed consent discussion indicating the minimal risks of leaving the floor, as the patient could ambulate independently and had no need for continuous monitoring. Finally, a physician’s order authorized the patient to be off the floor until 10 am. The patient returned to his room at 9:45 am and underwent a normal TEE, after which he was discharged home with outpatient follow-up.

The above scenario highlights the benefits of a comprehensive framework for patient movement practices that are transparent, fair, and systematic. Explicitly recognizing competing institutional and patient perspectives can prevent conflict and promote high-quality, safe, efficient, patient-centered care that only restricts the patient’s movement under specified and justifiable conditions. In developing strong hospital practices, institutions should refer to the relevant clinical and ethical standards and draw upon their institutional resources in risk management, clinical staff, and patient advocates.

Acknowledgments

The authors thank Dr. Neil Shapiro and Dr. David Chuquin for their constructive reviews of prior versions of this manuscript.

Disclosures

The authors have no financial conflicts of interest to disclose.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the U.S. Department of Veterans Affairs, the US Government, or the VA National Center for Ethics in Health Care.

Sepsis and heart failure are two common, costly, and deadly conditions. Among hospitalized Medicare patients, these conditions rank as the first and second most frequent principal diagnoses accounting for over $33 billion in spending across all payers.¹ One-third to one-half of all hospital deaths are estimated to occur in patients with sepsis,² and heart failure is listed as a contributing factor in over 10% of deaths in the United States.³

Previous research shows that evidence-based care decisions can impact the outcomes for these patients. For example, sepsis patients receiving intravenous fluids, blood cultures, broad-spectrum antibiotics, and lactate measurement within three hours of presentation have lower mortality rates.⁴ In heart failure, key interventions such as the appropriate use of ACE inhibitors, beta blockers, and referral to disease management programs reduce morbidity and mortality.⁵

However, rapid dissemination and adoption of evidence-based guidelines remain a challenge.^6,7 Policy makers have introduced incentives and penalties to support adoption, with varying levels of success. After four years of Centers for Medicare and Medicaid Services (CMS) penalties for hospitals with excess heart failure readmissions, only 21% performed well enough to avoid a penalty in 2017.⁸ CMS has been tracking sepsis bundle adherence as a core measure, but the rate in 2018 sat at just 54%.⁹ It is clear that new solutions are needed.¹⁰

AdventHealth (formerly Adventist Health System) is a growing, faith-based health system with hospitals across nine states. AdventHealth is a national leader in quality, safety, and patient satisfaction but is not immune to the challenges of delivering consistent, evidence-based care across an extensive network. To accelerate system-wide practice change, AdventHealth’s Office of Clinical Excellence (OCE) partnered with QURE Healthcare and Premier, Inc., to implement a physician engagement and care standardization collaboration involving nearly 100 hospitalists at eight facilities across five states.

This paper describes the results of the Adventist QURE Quality Project (AQQP), which used QURE’s validated, simulation-based measurement and feedback approach to engage hospitalists and standardize evidence-based practices for patients with sepsis and heart failure. We documented specific areas of variation identified in the simulations, how those practices changed through serial feedback, and the impact of those changes on real-world outcomes and costs.

Setting

AdventHealth has its headquarters in Altamonte Springs, Florida. It has facilities in nine states, which includes 48 hospitals. The OCE is comprised of physician leaders, project managers, and data analysts who sponsored the project from July 2016 through July 2018.

Study Participants

AdventHealth hospitals were invited to enroll their hospitalists in AQQP; eight AdventHealth hospitals across five states, representing 91 physicians and 16 nurse practitioners/physician’s assistants (APPs), agreed to participate. Participants included both AdventHealth-employed providers and contracted hospitalist groups. Provider participation was voluntary and not tied to financial incentives; however, participants received Continuing Medical Education credit and, if applicable, Maintenance of Certification points through the American Board of Internal Medicine.

Quasi-experimental Design

We used AdventHealth hospitals not participating in AQQP as a quasi-experimental control group. We leveraged this to measure the impact of concurrent secular effects, such as order sets and other system-wide training, that could also improve practice and outcomes in our study.

Study Objectives and Approach

The explicit goals of AQQP were to (1) measure how sepsis and heart failure patients are cared for across AdventHealth using Clinical Performance and Value (CPV) case simulations, (2) provide a forum for hospitalists to discuss clinical variation, and (3) reduce unneeded variation to improve quality and reduce cost. QURE developed 12 CPV simulated patient cases (six sepsis and six heart failure cases) with case-specific evidenced-based scoring criteria tied to national and Advent­Health evidence-based guidelines. AdventHealth order sets were embedded in the cases and accessible by participants as they cared for their patients.

CPV vignettes are simulated patient cases administered online, and have been validated as an accurate and responsive measure of clinical decision-making in both ambulatory^11-13 and inpatient settings.^14,15 Cases take 20-30 minutes each to complete and simulate a typical clinical encounter: taking the medical history, performing a physical examination, ordering tests, making the diagnosis, implementing initial treatment, and outlining a follow-up plan. Each case has predefined, evidence-based scoring criteria for each care domain. Cases and scoring criteria were reviewed by AdventHealth hospitalist program leaders and physician leaders in OCE. Provider responses were double-scored by trained physician abstractors. Scores range from 0%-100%, with higher scores reflecting greater alignment with best practice recommendations.

In each round of the project, AQQP participants completed two CPV cases, received personalized online feedback reports on their care decisions, and met (at the various sites and via web conference) for a facilitated group discussion on areas of high group variation. The personal feedback reports included the participant’s case score compared to the group average, a list of high-priority personalized improvement opportunities, a summary of the cost of unneeded care items, and links to relevant references. The group discussions focused on six items of high variation. Six total rounds of CPV measurement and feedback were held, one every four months.

At the study’s conclusion, we administered a brief satisfaction survey, asking providers to rate various aspects of the project on a five-point Likert scale.

Data

The study used two primary data sources: (1) care decisions made in the CPV simulated cases and (2) patient-level utilization data from Premier Inc.’s QualityAdvisor^TM (QA) data system. QA integrates quality, safety, and financial data from AdventHealth’s electronic medical record, claims data, charge master, and other resources. QualityAdvisor also calculates expected performance for critical measures, including cost per case and length of stay (LOS), based on a proprietary algorithm, which uses DRG classification, severity-of-illness, risk-of-mortality, and other patient risk factors. We pulled patient-level observed and expected data from AQQP qualifying physicians, defined as physicians participating in a majority of CPV measurement rounds. Of the 107 total hospitalists who participated, six providers did not participate in enough CPV rounds, and 22 providers left AdventHealth and could not be included in a patient-level impact analysis. These providers were replaced with 21 new hospitalists who were enrolled in the study and included in the CPV analysis but who did not have patient-level data before AQQP enrollment. Overall, 58 providers met the qualifying criteria to be included in the impact analysis. We compared their performance to a group of 96 hospitalists at facilities that were not participating in the project. Comparator facilities were selected based on quantitative measures of size and demographic matching the AQQP-facilities ensuring that both sets of hospitals (comparator and AQQP) exhibited similar levels of engagement with Advent- Health quality activities such as quality dashboard performance and order set usage. Baseline patient-level cost and LOS data covered from October 2015 to June 2016 and were re-measured annually throughout the project, from July 2016 to June 2018.

Statistical Analyses

We analyzed three primary outcomes: (1) general CPV-measured improvements in each round (scored against evidence-based scoring criteria); (2) disease-specific CPV improvements over each round; and (3) changes in patient-level outcomes and economic savings among AdventHealth pneumonia/sepsis and heart failure patients from the aforementioned improvements. We used Student’s t-test to analyze continuous outcome variables (including CPV, cost of care, and length of stay data) and Fisher’s exact test for binary outcome data. All statistical analyses were performed using Stata 14.2 (StataCorp LLC, College Station, Texas).

Baseline Characteristics and Assessment

A total of 107 AdventHealth hospitalists participated in this study (Appendix Table 1). 78.1% of these providers rated the organization’s focus on quality and lowering unnecessary costs as either “good” or “excellent,” but 78.8% also reported that variation in care provided by the group was “moderate” to “very high”.

At baseline, we observed high variability in the care of pneumonia patients with sepsis (pneumonia/sepsis) and heart failure patients as measured by the care decisions obtained in the CPV cases. The overall quality score, which is a weighted average across all domains, averaged 61.9% ± 10.5% for the group (Table 1). Disaggregating scores by condition, we found an average overall score of 59.4% ± 10.9% for pneumonia/sepsis and 64.4% ± 9.4% for heart failure. The diagnosis and treatment domains, which require the most clinical judgment, had the lowest average domain scores of 53.4% ± 20.9% and 51.6% ± 15.1%, respectively.

Changes in CPV Scores

To determine the impact of serial measurement and feedback, we compared performance in the first two rounds of the project with the last two rounds. We found that overall CPV quality scores showed a 4.8%-point absolute improvement (P < .001; Table 1). We saw improvements in all care domains, and those increases were significant in all but the workup (P = .470); the most significant increase was in diagnostic accuracy (+19.1%; P < .001).

By condition, scores showed similar, statistically significant overall improvements: +4.4%-points for pneumonia/sepsis (P = .001) and +5.5%-points for heart failure (P < .001) driven by increases in the diagnosis and treatment domains. For example, providers increased appropriate identification of HF severity by 21.5%-points (P < .001) and primary diagnosis of pneumonia/sepsis by 3.6%-points (P = .385).

In the treatment domain, which included clinical decisions related to initial management and follow-up care, there were several specific improvements. For HF, we found that performing all the essential treatment elements—prescribing diuretics, ACE inhibitors and beta blockers for appropriate patients—improved by 13.9%-points (P = .038); ordering VTE prophylaxis increased more than threefold, from 16.6% to 51.0% (P < .001; Table 2). For pneumonia/sepsis patients, absolute adherence to all four elements of the 3-hour sepsis bundle improved by 11.7%-points (P = .034). We also saw a decrease in low-value diagnostic workup items for patient cases in which the guidelines suggest they are not needed, such as urinary antigen testing, which declined by 14.6%-points (P = .001) and sputum cultures, which declined 26.4%-points (P = .004). In addition, outlining an evidence-based discharge plan including a follow-up visit, patient education and medication reconciliation improved, especially for pneumonia/sepsis patients by 24.3%-points (P < .001).

Patient-Level Impact on LOS and Cost Per Case

We took advantage of the quasi-experimental design, in which only a portion of AdventHealth facilities participated in the project, to compare patient-level results from AQQP-participating physicians against the engagement-matched cohort of hospitalists at nonparticipating AdventHealth facilities. We adjusted for potential differences in patient-level case mix between the two groups by comparing the observed/expected (O/E) LOS and cost per case ratios for pneumonia/sepsis and heart failure patients.

At baseline, AQQP-hospitalists performed better on geometric LOS versus the comparator group (O/E of 1.13 vs 1.22; P = .006) but at about the same on cost per case (O/E of 1.16 vs 1.14; P = .390). Throughout the project, as patient volumes and expected per patient costs rose for both groups, O/E ratios improved among both AQQP and non-AQQP providers.

To set apart the contribution of system-wide improvements from the AQQP project-specific impacts, we applied the O/E improvement rates seen in the comparator group to the AQQP group baseline performance. We then compared that to the actual changes seen in the AQQP throughout the project to see if there was any additional benefit from the simulation measurement and feedback (Figure).

Provider Satisfaction

At the project conclusion, we administered a brief survey. Participants were asked to rate aspects of the project (a five-point Likert scale with five being the highest), and 24 responded. The mean ratings of the relevance of the project to their practice and the overall quality of the material were 4.5 and 4.2, respectively. Providers found the individual feedback reports (3.9) slightly more helpful than the webcast group discussions (3.7; Appendix Table 2 ).

As health systems expand, the opportunity to standardize clinical practice within a system has the potential to enhance patient care and lower costs. However, achieving these goals is challenging when providers are dispersed across geographically separated sites and clinical decision-making is difficult to measure in a standardized way.^16,17 We brought together over 100 physicians and APPs from eight different-sized hospitals in five different states to prospectively determine if we could improve care using a standardized measurement and feedback system. At baseline, we found that care varied dramatically among providers. Care varied in terms of diagnostic accuracy and treatment, which directly relate to care quality and outcomes.⁴ After serial measurement and feedback, we saw reductions in unnecessary testing, more guideline-based treatment decisions, and better discharge planning in the clinical vignettes.

We confirmed that changes in CPV-measured practice translated into lower costs and shorter LOS at the patient level. We further validated the improvements through a quasi-experimental design that compared these changes to those at nonparticipating AdventHealth facilities. We saw more significant cost reductions and decreases in LOS in the simulation-based measurement and feedback cohort with the biggest impact early on. The overall savings to the system, attributable specifically to the AQQP approach, is estimated to be $2.4 million.

One advantage of the online case simulation approach is the ability to bring geographically remote sites together in a shared quality-of-care discussion. The interventions specifically sought to remove barriers between facilities. For example, individual feedback reports allowed providers to see how they compare with providers at other AdventHealth facilities and webcast results discussions enable providers across facilities to discuss specific care decisions.

There were several limitations to the study. While the quasi-experimental design allowed us to make informative comparisons between AQQP-participating facilities and nonparticipating facilities, the assignments were not random, and participants were generally from higher performing hospital medicine groups. The determination of secular versus CPV-related improvement is confounded by other system improvement initiatives that may have impacted cost and LOS results. This is mitigated by the observation that facilities that opted to participate performed better at baseline in risk-adjusted LOS but slightly worse in cost per case, indicating that baseline differences were not dramatic. While both groups improved over time, the QURE measurement and feedback approach led to larger and more rapid gains than those seen in the comparator group. However, we could not exclude the potential that project participation at the site level was biased to those groups disposed to performance improvement. In addition, our patient-level data analysis was limited to the metrics available and did not allow us to directly compare patient-level performance across the plethora of clinically relevant CPV data that showed improvement. Our inpatient cost per case analysis showed significant savings for the system but did not include all potentially favorable economic impacts such as lower follow-up care costs for patients, more accurate reimbursement through better coding or fewer lost days of productivity.

With continued consolidation in healthcare and broader health systems spanning multiple geographies, new tools are needed to support standardized, evidence-based care across sites. This standardization is especially important, both clinically and financially, for high-volume, high-cost diseases such as sepsis and heart failure. However, changing practice cannot happen without collaborative engagement with providers. Standardized patient vignettes are an opportunity to measure and provide feedback in a systematic way that engages providers and is particularly well-suited to large systems and common clinical conditions. This analysis, from a real-world study, shows that an approach that standardizes care and lowers costs may be particularly helpful for large systems needing to bring disparate sites together as they concurrently move toward value-based payment.

Disclosures

QURE, LLC, whose intellectual property was used to prepare the cases and collect the data, was contracted by AdventHealth. Otherwise, any of the study authors report no potential conflicts to disclose.

Funding

This work was funded by a contract between AdventHealth (formerly Adventist Health System) and QURE, LLC.

Sepsis and heart failure are two common, costly, and deadly conditions. Among hospitalized Medicare patients, these conditions rank as the first and second most frequent principal diagnoses accounting for over $33 billion in spending across all payers.¹ One-third to one-half of all hospital deaths are estimated to occur in patients with sepsis,² and heart failure is listed as a contributing factor in over 10% of deaths in the United States.³

Previous research shows that evidence-based care decisions can impact the outcomes for these patients. For example, sepsis patients receiving intravenous fluids, blood cultures, broad-spectrum antibiotics, and lactate measurement within three hours of presentation have lower mortality rates.⁴ In heart failure, key interventions such as the appropriate use of ACE inhibitors, beta blockers, and referral to disease management programs reduce morbidity and mortality.⁵

However, rapid dissemination and adoption of evidence-based guidelines remain a challenge.^6,7 Policy makers have introduced incentives and penalties to support adoption, with varying levels of success. After four years of Centers for Medicare and Medicaid Services (CMS) penalties for hospitals with excess heart failure readmissions, only 21% performed well enough to avoid a penalty in 2017.⁸ CMS has been tracking sepsis bundle adherence as a core measure, but the rate in 2018 sat at just 54%.⁹ It is clear that new solutions are needed.¹⁰

AdventHealth (formerly Adventist Health System) is a growing, faith-based health system with hospitals across nine states. AdventHealth is a national leader in quality, safety, and patient satisfaction but is not immune to the challenges of delivering consistent, evidence-based care across an extensive network. To accelerate system-wide practice change, AdventHealth’s Office of Clinical Excellence (OCE) partnered with QURE Healthcare and Premier, Inc., to implement a physician engagement and care standardization collaboration involving nearly 100 hospitalists at eight facilities across five states.

This paper describes the results of the Adventist QURE Quality Project (AQQP), which used QURE’s validated, simulation-based measurement and feedback approach to engage hospitalists and standardize evidence-based practices for patients with sepsis and heart failure. We documented specific areas of variation identified in the simulations, how those practices changed through serial feedback, and the impact of those changes on real-world outcomes and costs.

Setting

AdventHealth has its headquarters in Altamonte Springs, Florida. It has facilities in nine states, which includes 48 hospitals. The OCE is comprised of physician leaders, project managers, and data analysts who sponsored the project from July 2016 through July 2018.

Study Participants

AdventHealth hospitals were invited to enroll their hospitalists in AQQP; eight AdventHealth hospitals across five states, representing 91 physicians and 16 nurse practitioners/physician’s assistants (APPs), agreed to participate. Participants included both AdventHealth-employed providers and contracted hospitalist groups. Provider participation was voluntary and not tied to financial incentives; however, participants received Continuing Medical Education credit and, if applicable, Maintenance of Certification points through the American Board of Internal Medicine.

Quasi-experimental Design

We used AdventHealth hospitals not participating in AQQP as a quasi-experimental control group. We leveraged this to measure the impact of concurrent secular effects, such as order sets and other system-wide training, that could also improve practice and outcomes in our study.

Study Objectives and Approach

The explicit goals of AQQP were to (1) measure how sepsis and heart failure patients are cared for across AdventHealth using Clinical Performance and Value (CPV) case simulations, (2) provide a forum for hospitalists to discuss clinical variation, and (3) reduce unneeded variation to improve quality and reduce cost. QURE developed 12 CPV simulated patient cases (six sepsis and six heart failure cases) with case-specific evidenced-based scoring criteria tied to national and Advent­Health evidence-based guidelines. AdventHealth order sets were embedded in the cases and accessible by participants as they cared for their patients.

CPV vignettes are simulated patient cases administered online, and have been validated as an accurate and responsive measure of clinical decision-making in both ambulatory^11-13 and inpatient settings.^14,15 Cases take 20-30 minutes each to complete and simulate a typical clinical encounter: taking the medical history, performing a physical examination, ordering tests, making the diagnosis, implementing initial treatment, and outlining a follow-up plan. Each case has predefined, evidence-based scoring criteria for each care domain. Cases and scoring criteria were reviewed by AdventHealth hospitalist program leaders and physician leaders in OCE. Provider responses were double-scored by trained physician abstractors. Scores range from 0%-100%, with higher scores reflecting greater alignment with best practice recommendations.

In each round of the project, AQQP participants completed two CPV cases, received personalized online feedback reports on their care decisions, and met (at the various sites and via web conference) for a facilitated group discussion on areas of high group variation. The personal feedback reports included the participant’s case score compared to the group average, a list of high-priority personalized improvement opportunities, a summary of the cost of unneeded care items, and links to relevant references. The group discussions focused on six items of high variation. Six total rounds of CPV measurement and feedback were held, one every four months.

At the study’s conclusion, we administered a brief satisfaction survey, asking providers to rate various aspects of the project on a five-point Likert scale.

Data

The study used two primary data sources: (1) care decisions made in the CPV simulated cases and (2) patient-level utilization data from Premier Inc.’s QualityAdvisor^TM (QA) data system. QA integrates quality, safety, and financial data from AdventHealth’s electronic medical record, claims data, charge master, and other resources. QualityAdvisor also calculates expected performance for critical measures, including cost per case and length of stay (LOS), based on a proprietary algorithm, which uses DRG classification, severity-of-illness, risk-of-mortality, and other patient risk factors. We pulled patient-level observed and expected data from AQQP qualifying physicians, defined as physicians participating in a majority of CPV measurement rounds. Of the 107 total hospitalists who participated, six providers did not participate in enough CPV rounds, and 22 providers left AdventHealth and could not be included in a patient-level impact analysis. These providers were replaced with 21 new hospitalists who were enrolled in the study and included in the CPV analysis but who did not have patient-level data before AQQP enrollment. Overall, 58 providers met the qualifying criteria to be included in the impact analysis. We compared their performance to a group of 96 hospitalists at facilities that were not participating in the project. Comparator facilities were selected based on quantitative measures of size and demographic matching the AQQP-facilities ensuring that both sets of hospitals (comparator and AQQP) exhibited similar levels of engagement with Advent- Health quality activities such as quality dashboard performance and order set usage. Baseline patient-level cost and LOS data covered from October 2015 to June 2016 and were re-measured annually throughout the project, from July 2016 to June 2018.

Statistical Analyses

We analyzed three primary outcomes: (1) general CPV-measured improvements in each round (scored against evidence-based scoring criteria); (2) disease-specific CPV improvements over each round; and (3) changes in patient-level outcomes and economic savings among AdventHealth pneumonia/sepsis and heart failure patients from the aforementioned improvements. We used Student’s t-test to analyze continuous outcome variables (including CPV, cost of care, and length of stay data) and Fisher’s exact test for binary outcome data. All statistical analyses were performed using Stata 14.2 (StataCorp LLC, College Station, Texas).

Baseline Characteristics and Assessment

A total of 107 AdventHealth hospitalists participated in this study (Appendix Table 1). 78.1% of these providers rated the organization’s focus on quality and lowering unnecessary costs as either “good” or “excellent,” but 78.8% also reported that variation in care provided by the group was “moderate” to “very high”.

At baseline, we observed high variability in the care of pneumonia patients with sepsis (pneumonia/sepsis) and heart failure patients as measured by the care decisions obtained in the CPV cases. The overall quality score, which is a weighted average across all domains, averaged 61.9% ± 10.5% for the group (Table 1). Disaggregating scores by condition, we found an average overall score of 59.4% ± 10.9% for pneumonia/sepsis and 64.4% ± 9.4% for heart failure. The diagnosis and treatment domains, which require the most clinical judgment, had the lowest average domain scores of 53.4% ± 20.9% and 51.6% ± 15.1%, respectively.

Changes in CPV Scores

To determine the impact of serial measurement and feedback, we compared performance in the first two rounds of the project with the last two rounds. We found that overall CPV quality scores showed a 4.8%-point absolute improvement (P < .001; Table 1). We saw improvements in all care domains, and those increases were significant in all but the workup (P = .470); the most significant increase was in diagnostic accuracy (+19.1%; P < .001).

By condition, scores showed similar, statistically significant overall improvements: +4.4%-points for pneumonia/sepsis (P = .001) and +5.5%-points for heart failure (P < .001) driven by increases in the diagnosis and treatment domains. For example, providers increased appropriate identification of HF severity by 21.5%-points (P < .001) and primary diagnosis of pneumonia/sepsis by 3.6%-points (P = .385).

In the treatment domain, which included clinical decisions related to initial management and follow-up care, there were several specific improvements. For HF, we found that performing all the essential treatment elements—prescribing diuretics, ACE inhibitors and beta blockers for appropriate patients—improved by 13.9%-points (P = .038); ordering VTE prophylaxis increased more than threefold, from 16.6% to 51.0% (P < .001; Table 2). For pneumonia/sepsis patients, absolute adherence to all four elements of the 3-hour sepsis bundle improved by 11.7%-points (P = .034). We also saw a decrease in low-value diagnostic workup items for patient cases in which the guidelines suggest they are not needed, such as urinary antigen testing, which declined by 14.6%-points (P = .001) and sputum cultures, which declined 26.4%-points (P = .004). In addition, outlining an evidence-based discharge plan including a follow-up visit, patient education and medication reconciliation improved, especially for pneumonia/sepsis patients by 24.3%-points (P < .001).

Patient-Level Impact on LOS and Cost Per Case

We took advantage of the quasi-experimental design, in which only a portion of AdventHealth facilities participated in the project, to compare patient-level results from AQQP-participating physicians against the engagement-matched cohort of hospitalists at nonparticipating AdventHealth facilities. We adjusted for potential differences in patient-level case mix between the two groups by comparing the observed/expected (O/E) LOS and cost per case ratios for pneumonia/sepsis and heart failure patients.

At baseline, AQQP-hospitalists performed better on geometric LOS versus the comparator group (O/E of 1.13 vs 1.22; P = .006) but at about the same on cost per case (O/E of 1.16 vs 1.14; P = .390). Throughout the project, as patient volumes and expected per patient costs rose for both groups, O/E ratios improved among both AQQP and non-AQQP providers.

To set apart the contribution of system-wide improvements from the AQQP project-specific impacts, we applied the O/E improvement rates seen in the comparator group to the AQQP group baseline performance. We then compared that to the actual changes seen in the AQQP throughout the project to see if there was any additional benefit from the simulation measurement and feedback (Figure).

Provider Satisfaction

At the project conclusion, we administered a brief survey. Participants were asked to rate aspects of the project (a five-point Likert scale with five being the highest), and 24 responded. The mean ratings of the relevance of the project to their practice and the overall quality of the material were 4.5 and 4.2, respectively. Providers found the individual feedback reports (3.9) slightly more helpful than the webcast group discussions (3.7; Appendix Table 2 ).

As health systems expand, the opportunity to standardize clinical practice within a system has the potential to enhance patient care and lower costs. However, achieving these goals is challenging when providers are dispersed across geographically separated sites and clinical decision-making is difficult to measure in a standardized way.^16,17 We brought together over 100 physicians and APPs from eight different-sized hospitals in five different states to prospectively determine if we could improve care using a standardized measurement and feedback system. At baseline, we found that care varied dramatically among providers. Care varied in terms of diagnostic accuracy and treatment, which directly relate to care quality and outcomes.⁴ After serial measurement and feedback, we saw reductions in unnecessary testing, more guideline-based treatment decisions, and better discharge planning in the clinical vignettes.

We confirmed that changes in CPV-measured practice translated into lower costs and shorter LOS at the patient level. We further validated the improvements through a quasi-experimental design that compared these changes to those at nonparticipating AdventHealth facilities. We saw more significant cost reductions and decreases in LOS in the simulation-based measurement and feedback cohort with the biggest impact early on. The overall savings to the system, attributable specifically to the AQQP approach, is estimated to be $2.4 million.

One advantage of the online case simulation approach is the ability to bring geographically remote sites together in a shared quality-of-care discussion. The interventions specifically sought to remove barriers between facilities. For example, individual feedback reports allowed providers to see how they compare with providers at other AdventHealth facilities and webcast results discussions enable providers across facilities to discuss specific care decisions.

There were several limitations to the study. While the quasi-experimental design allowed us to make informative comparisons between AQQP-participating facilities and nonparticipating facilities, the assignments were not random, and participants were generally from higher performing hospital medicine groups. The determination of secular versus CPV-related improvement is confounded by other system improvement initiatives that may have impacted cost and LOS results. This is mitigated by the observation that facilities that opted to participate performed better at baseline in risk-adjusted LOS but slightly worse in cost per case, indicating that baseline differences were not dramatic. While both groups improved over time, the QURE measurement and feedback approach led to larger and more rapid gains than those seen in the comparator group. However, we could not exclude the potential that project participation at the site level was biased to those groups disposed to performance improvement. In addition, our patient-level data analysis was limited to the metrics available and did not allow us to directly compare patient-level performance across the plethora of clinically relevant CPV data that showed improvement. Our inpatient cost per case analysis showed significant savings for the system but did not include all potentially favorable economic impacts such as lower follow-up care costs for patients, more accurate reimbursement through better coding or fewer lost days of productivity.

With continued consolidation in healthcare and broader health systems spanning multiple geographies, new tools are needed to support standardized, evidence-based care across sites. This standardization is especially important, both clinically and financially, for high-volume, high-cost diseases such as sepsis and heart failure. However, changing practice cannot happen without collaborative engagement with providers. Standardized patient vignettes are an opportunity to measure and provide feedback in a systematic way that engages providers and is particularly well-suited to large systems and common clinical conditions. This analysis, from a real-world study, shows that an approach that standardizes care and lowers costs may be particularly helpful for large systems needing to bring disparate sites together as they concurrently move toward value-based payment.

Disclosures

QURE, LLC, whose intellectual property was used to prepare the cases and collect the data, was contracted by AdventHealth. Otherwise, any of the study authors report no potential conflicts to disclose.

Funding

This work was funded by a contract between AdventHealth (formerly Adventist Health System) and QURE, LLC.

Sepsis and heart failure are two common, costly, and deadly conditions. Among hospitalized Medicare patients, these conditions rank as the first and second most frequent principal diagnoses accounting for over $33 billion in spending across all payers.¹ One-third to one-half of all hospital deaths are estimated to occur in patients with sepsis,² and heart failure is listed as a contributing factor in over 10% of deaths in the United States.³

Previous research shows that evidence-based care decisions can impact the outcomes for these patients. For example, sepsis patients receiving intravenous fluids, blood cultures, broad-spectrum antibiotics, and lactate measurement within three hours of presentation have lower mortality rates.⁴ In heart failure, key interventions such as the appropriate use of ACE inhibitors, beta blockers, and referral to disease management programs reduce morbidity and mortality.⁵

However, rapid dissemination and adoption of evidence-based guidelines remain a challenge.^6,7 Policy makers have introduced incentives and penalties to support adoption, with varying levels of success. After four years of Centers for Medicare and Medicaid Services (CMS) penalties for hospitals with excess heart failure readmissions, only 21% performed well enough to avoid a penalty in 2017.⁸ CMS has been tracking sepsis bundle adherence as a core measure, but the rate in 2018 sat at just 54%.⁹ It is clear that new solutions are needed.¹⁰

AdventHealth (formerly Adventist Health System) is a growing, faith-based health system with hospitals across nine states. AdventHealth is a national leader in quality, safety, and patient satisfaction but is not immune to the challenges of delivering consistent, evidence-based care across an extensive network. To accelerate system-wide practice change, AdventHealth’s Office of Clinical Excellence (OCE) partnered with QURE Healthcare and Premier, Inc., to implement a physician engagement and care standardization collaboration involving nearly 100 hospitalists at eight facilities across five states.

This paper describes the results of the Adventist QURE Quality Project (AQQP), which used QURE’s validated, simulation-based measurement and feedback approach to engage hospitalists and standardize evidence-based practices for patients with sepsis and heart failure. We documented specific areas of variation identified in the simulations, how those practices changed through serial feedback, and the impact of those changes on real-world outcomes and costs.

Setting

AdventHealth has its headquarters in Altamonte Springs, Florida. It has facilities in nine states, which includes 48 hospitals. The OCE is comprised of physician leaders, project managers, and data analysts who sponsored the project from July 2016 through July 2018.

Study Participants

AdventHealth hospitals were invited to enroll their hospitalists in AQQP; eight AdventHealth hospitals across five states, representing 91 physicians and 16 nurse practitioners/physician’s assistants (APPs), agreed to participate. Participants included both AdventHealth-employed providers and contracted hospitalist groups. Provider participation was voluntary and not tied to financial incentives; however, participants received Continuing Medical Education credit and, if applicable, Maintenance of Certification points through the American Board of Internal Medicine.

Quasi-experimental Design

We used AdventHealth hospitals not participating in AQQP as a quasi-experimental control group. We leveraged this to measure the impact of concurrent secular effects, such as order sets and other system-wide training, that could also improve practice and outcomes in our study.

Study Objectives and Approach

The explicit goals of AQQP were to (1) measure how sepsis and heart failure patients are cared for across AdventHealth using Clinical Performance and Value (CPV) case simulations, (2) provide a forum for hospitalists to discuss clinical variation, and (3) reduce unneeded variation to improve quality and reduce cost. QURE developed 12 CPV simulated patient cases (six sepsis and six heart failure cases) with case-specific evidenced-based scoring criteria tied to national and Advent­Health evidence-based guidelines. AdventHealth order sets were embedded in the cases and accessible by participants as they cared for their patients.

CPV vignettes are simulated patient cases administered online, and have been validated as an accurate and responsive measure of clinical decision-making in both ambulatory^11-13 and inpatient settings.^14,15 Cases take 20-30 minutes each to complete and simulate a typical clinical encounter: taking the medical history, performing a physical examination, ordering tests, making the diagnosis, implementing initial treatment, and outlining a follow-up plan. Each case has predefined, evidence-based scoring criteria for each care domain. Cases and scoring criteria were reviewed by AdventHealth hospitalist program leaders and physician leaders in OCE. Provider responses were double-scored by trained physician abstractors. Scores range from 0%-100%, with higher scores reflecting greater alignment with best practice recommendations.

In each round of the project, AQQP participants completed two CPV cases, received personalized online feedback reports on their care decisions, and met (at the various sites and via web conference) for a facilitated group discussion on areas of high group variation. The personal feedback reports included the participant’s case score compared to the group average, a list of high-priority personalized improvement opportunities, a summary of the cost of unneeded care items, and links to relevant references. The group discussions focused on six items of high variation. Six total rounds of CPV measurement and feedback were held, one every four months.

At the study’s conclusion, we administered a brief satisfaction survey, asking providers to rate various aspects of the project on a five-point Likert scale.

Data

The study used two primary data sources: (1) care decisions made in the CPV simulated cases and (2) patient-level utilization data from Premier Inc.’s QualityAdvisor^TM (QA) data system. QA integrates quality, safety, and financial data from AdventHealth’s electronic medical record, claims data, charge master, and other resources. QualityAdvisor also calculates expected performance for critical measures, including cost per case and length of stay (LOS), based on a proprietary algorithm, which uses DRG classification, severity-of-illness, risk-of-mortality, and other patient risk factors. We pulled patient-level observed and expected data from AQQP qualifying physicians, defined as physicians participating in a majority of CPV measurement rounds. Of the 107 total hospitalists who participated, six providers did not participate in enough CPV rounds, and 22 providers left AdventHealth and could not be included in a patient-level impact analysis. These providers were replaced with 21 new hospitalists who were enrolled in the study and included in the CPV analysis but who did not have patient-level data before AQQP enrollment. Overall, 58 providers met the qualifying criteria to be included in the impact analysis. We compared their performance to a group of 96 hospitalists at facilities that were not participating in the project. Comparator facilities were selected based on quantitative measures of size and demographic matching the AQQP-facilities ensuring that both sets of hospitals (comparator and AQQP) exhibited similar levels of engagement with Advent- Health quality activities such as quality dashboard performance and order set usage. Baseline patient-level cost and LOS data covered from October 2015 to June 2016 and were re-measured annually throughout the project, from July 2016 to June 2018.

Statistical Analyses

We analyzed three primary outcomes: (1) general CPV-measured improvements in each round (scored against evidence-based scoring criteria); (2) disease-specific CPV improvements over each round; and (3) changes in patient-level outcomes and economic savings among AdventHealth pneumonia/sepsis and heart failure patients from the aforementioned improvements. We used Student’s t-test to analyze continuous outcome variables (including CPV, cost of care, and length of stay data) and Fisher’s exact test for binary outcome data. All statistical analyses were performed using Stata 14.2 (StataCorp LLC, College Station, Texas).

Baseline Characteristics and Assessment

A total of 107 AdventHealth hospitalists participated in this study (Appendix Table 1). 78.1% of these providers rated the organization’s focus on quality and lowering unnecessary costs as either “good” or “excellent,” but 78.8% also reported that variation in care provided by the group was “moderate” to “very high”.

At baseline, we observed high variability in the care of pneumonia patients with sepsis (pneumonia/sepsis) and heart failure patients as measured by the care decisions obtained in the CPV cases. The overall quality score, which is a weighted average across all domains, averaged 61.9% ± 10.5% for the group (Table 1). Disaggregating scores by condition, we found an average overall score of 59.4% ± 10.9% for pneumonia/sepsis and 64.4% ± 9.4% for heart failure. The diagnosis and treatment domains, which require the most clinical judgment, had the lowest average domain scores of 53.4% ± 20.9% and 51.6% ± 15.1%, respectively.

Changes in CPV Scores

To determine the impact of serial measurement and feedback, we compared performance in the first two rounds of the project with the last two rounds. We found that overall CPV quality scores showed a 4.8%-point absolute improvement (P < .001; Table 1). We saw improvements in all care domains, and those increases were significant in all but the workup (P = .470); the most significant increase was in diagnostic accuracy (+19.1%; P < .001).

By condition, scores showed similar, statistically significant overall improvements: +4.4%-points for pneumonia/sepsis (P = .001) and +5.5%-points for heart failure (P < .001) driven by increases in the diagnosis and treatment domains. For example, providers increased appropriate identification of HF severity by 21.5%-points (P < .001) and primary diagnosis of pneumonia/sepsis by 3.6%-points (P = .385).

In the treatment domain, which included clinical decisions related to initial management and follow-up care, there were several specific improvements. For HF, we found that performing all the essential treatment elements—prescribing diuretics, ACE inhibitors and beta blockers for appropriate patients—improved by 13.9%-points (P = .038); ordering VTE prophylaxis increased more than threefold, from 16.6% to 51.0% (P < .001; Table 2). For pneumonia/sepsis patients, absolute adherence to all four elements of the 3-hour sepsis bundle improved by 11.7%-points (P = .034). We also saw a decrease in low-value diagnostic workup items for patient cases in which the guidelines suggest they are not needed, such as urinary antigen testing, which declined by 14.6%-points (P = .001) and sputum cultures, which declined 26.4%-points (P = .004). In addition, outlining an evidence-based discharge plan including a follow-up visit, patient education and medication reconciliation improved, especially for pneumonia/sepsis patients by 24.3%-points (P < .001).

Patient-Level Impact on LOS and Cost Per Case

We took advantage of the quasi-experimental design, in which only a portion of AdventHealth facilities participated in the project, to compare patient-level results from AQQP-participating physicians against the engagement-matched cohort of hospitalists at nonparticipating AdventHealth facilities. We adjusted for potential differences in patient-level case mix between the two groups by comparing the observed/expected (O/E) LOS and cost per case ratios for pneumonia/sepsis and heart failure patients.

At baseline, AQQP-hospitalists performed better on geometric LOS versus the comparator group (O/E of 1.13 vs 1.22; P = .006) but at about the same on cost per case (O/E of 1.16 vs 1.14; P = .390). Throughout the project, as patient volumes and expected per patient costs rose for both groups, O/E ratios improved among both AQQP and non-AQQP providers.

To set apart the contribution of system-wide improvements from the AQQP project-specific impacts, we applied the O/E improvement rates seen in the comparator group to the AQQP group baseline performance. We then compared that to the actual changes seen in the AQQP throughout the project to see if there was any additional benefit from the simulation measurement and feedback (Figure).

Provider Satisfaction

At the project conclusion, we administered a brief survey. Participants were asked to rate aspects of the project (a five-point Likert scale with five being the highest), and 24 responded. The mean ratings of the relevance of the project to their practice and the overall quality of the material were 4.5 and 4.2, respectively. Providers found the individual feedback reports (3.9) slightly more helpful than the webcast group discussions (3.7; Appendix Table 2 ).

As health systems expand, the opportunity to standardize clinical practice within a system has the potential to enhance patient care and lower costs. However, achieving these goals is challenging when providers are dispersed across geographically separated sites and clinical decision-making is difficult to measure in a standardized way.^16,17 We brought together over 100 physicians and APPs from eight different-sized hospitals in five different states to prospectively determine if we could improve care using a standardized measurement and feedback system. At baseline, we found that care varied dramatically among providers. Care varied in terms of diagnostic accuracy and treatment, which directly relate to care quality and outcomes.⁴ After serial measurement and feedback, we saw reductions in unnecessary testing, more guideline-based treatment decisions, and better discharge planning in the clinical vignettes.

We confirmed that changes in CPV-measured practice translated into lower costs and shorter LOS at the patient level. We further validated the improvements through a quasi-experimental design that compared these changes to those at nonparticipating AdventHealth facilities. We saw more significant cost reductions and decreases in LOS in the simulation-based measurement and feedback cohort with the biggest impact early on. The overall savings to the system, attributable specifically to the AQQP approach, is estimated to be $2.4 million.

One advantage of the online case simulation approach is the ability to bring geographically remote sites together in a shared quality-of-care discussion. The interventions specifically sought to remove barriers between facilities. For example, individual feedback reports allowed providers to see how they compare with providers at other AdventHealth facilities and webcast results discussions enable providers across facilities to discuss specific care decisions.

There were several limitations to the study. While the quasi-experimental design allowed us to make informative comparisons between AQQP-participating facilities and nonparticipating facilities, the assignments were not random, and participants were generally from higher performing hospital medicine groups. The determination of secular versus CPV-related improvement is confounded by other system improvement initiatives that may have impacted cost and LOS results. This is mitigated by the observation that facilities that opted to participate performed better at baseline in risk-adjusted LOS but slightly worse in cost per case, indicating that baseline differences were not dramatic. While both groups improved over time, the QURE measurement and feedback approach led to larger and more rapid gains than those seen in the comparator group. However, we could not exclude the potential that project participation at the site level was biased to those groups disposed to performance improvement. In addition, our patient-level data analysis was limited to the metrics available and did not allow us to directly compare patient-level performance across the plethora of clinically relevant CPV data that showed improvement. Our inpatient cost per case analysis showed significant savings for the system but did not include all potentially favorable economic impacts such as lower follow-up care costs for patients, more accurate reimbursement through better coding or fewer lost days of productivity.

With continued consolidation in healthcare and broader health systems spanning multiple geographies, new tools are needed to support standardized, evidence-based care across sites. This standardization is especially important, both clinically and financially, for high-volume, high-cost diseases such as sepsis and heart failure. However, changing practice cannot happen without collaborative engagement with providers. Standardized patient vignettes are an opportunity to measure and provide feedback in a systematic way that engages providers and is particularly well-suited to large systems and common clinical conditions. This analysis, from a real-world study, shows that an approach that standardizes care and lowers costs may be particularly helpful for large systems needing to bring disparate sites together as they concurrently move toward value-based payment.

Disclosures

QURE, LLC, whose intellectual property was used to prepare the cases and collect the data, was contracted by AdventHealth. Otherwise, any of the study authors report no potential conflicts to disclose.

Funding

This work was funded by a contract between AdventHealth (formerly Adventist Health System) and QURE, LLC.

Approximately 60,000 hospitalists were working in the United States in 2018.¹ Hospitalist groups work collaboratively because of the shiftwork required for 24/7 patient coverage, and first-rate clinical documentation is essential for quality care.² Thoughtful clinical documentation not only transmits one provider’s clinical reasoning to other providers but is a professional responsibility.³ Hospitalists spend two-thirds of their time in indirect patient-care activities and approximately one quarter of their time on documentation in electronic health records (EHRs).⁴ Despite documentation occupying a substantial portion of the clinician’s time, published literature on the best practices for the documentation of clinical reasoning in hospital medicine or its assessment remains scant.^5-7

Clinical reasoning involves establishing a diagnosis and developing a therapeutic plan that fits the unique circumstances and needs of the patient.⁸ Inpatient providers who admit patients to the hospital end the admission note with their assessment and plan (A&P) after reflecting about a patient’s presenting illness. The A&P generally represents the interpretations, deductions, and clinical reasoning of the inpatient providers; this is the section of the note that fellow physicians concentrate on over others.⁹ The documentation of clinical reasoning in the A&P allows for many to consider how the recorded interpretations relate to their own elucidations resulting in distributed cognition.¹⁰

Disorganized documentation can contribute to cognitive overload and impede thoughtful consideration about the clinical presentation.³ The assessment of clinical documentation may translate into reduced medical errors and improved note quality.^11,12 Studies that have formally evaluated the documentation of clinical reasoning have focused exclusively on medical students.^13-15 The nonexistence of a detailed rubric for evaluating clinical reasoning in the A&Ps of hospitalists represents a missed opportunity for evaluating what hospitalists “do”; if this evolves into a mechanism for offering formative feedback, such professional development would impact the highest level of Miller’s assessment pyramid.¹⁶ We therefore undertook this study to establish a metric to assess the hospitalist providers’ documentation of clinical reasoning in the A&P of an admission note.

Study Design, Setting, and Subjects

This was a retrospective study that reviewed the admission notes of hospitalists for patients admitted over the period of January 2014 and October 2017 at three hospitals in Maryland. One is a community hospital (Hospital A) and two are academic medical centers (Hospital B and Hospital C). Even though these three hospitals are part of one health system, they have distinct cultures and leadership, serve different populations, and are staffed by different provider teams.

The notes of physicians working for the hospitalist groups at each of the three hospitals were the focus of the analysis in this study.

Development of the Documentation Assessment Rubric

A team was assembled to develop the Clinical Reasoning in Admission Note Assessment & PLan (CRANAPL) tool. The CRANAPL was designed to assess the comprehensiveness and thoughtfulness of the clinical reasoning documented in the A&P sections of the notes of patients who were admitted to the hospital with an acute illness. Validity evidence for CRANAPL was summarized on the basis of Messick’s unified validity framework by using four of the five sources of validity: content, response process, internal structure, and relations to other variables.¹⁷

Content Validity

The development team consisted of members who have an average of 10 years of clinical experience in hospital medicine; have studied clinical excellence and clinical reasoning; and have expertise in feedback, assessment, and professional development.^18-22 The development of the CRANAPL tool by the team was informed by a review of the clinical reasoning literature, with particular attention paid to the standards and competencies outlined by the Liaison Committee on Medical Education, the Association of American Medical Colleges, the Accreditation Council on Graduate Medical Education, the Internal Medicine Milestone Project, and the Society of Hospital Medicine.^23-26 For each of these parties, diagnostic reasoning and its impact on clinical decision-making are considered to be a core competency. Several works that heavily influenced the CRANAPL tool’s development were Baker’s Interpretive Summary, Differential Diagnosis, Explanation of Reasoning, And Alternatives (IDEA) assessment tool;¹⁴ King’s Pediatric History and Physical Exam Evaluation (P-HAPEE) rubric;¹⁵ and three other studies related to diagnostic reasoning.^16,27,28 These manuscripts and other works substantively informed the preliminary behavioral-based anchors that formed the initial foundation for the tool under development. The CRANAPL tool was shown to colleagues at other institutions who are leaders on clinical reasoning and was presented at academic conferences in the Division of General Internal Medicine and the Division of Hospital Medicine of our institution. Feedback resulted in iterative revisions. The aforementioned methods established content validity evidence for the CRANAPL tool.

Response Process Validity

Several of the authors pilot-tested earlier iterations on admission notes that were excluded from the sample when refining the CRANAPL tool. The weaknesses and sources of confusion with specific items were addressed by scoring 10 A&Ps individually and then comparing data captured on the tool. This cycle was repeated three times for the iterative enhancement and finalization of the CRANAPL tool. On several occasions when two authors were piloting the near-final CRANAPL tool, a third author interviewed each of the two authors about reactivity while assessing individual items and exploring with probes how their own clinical documentation practices were being considered when scoring the notes. The reasonable and thoughtful answers provided by the two authors as they explained and justified the scores they were selecting during the pilot testing served to confer response process validity evidence.

Finalizing the CRANAPL Tool

The nine-item CRANAPL tool includes elements for problem representation, leading diagnosis, uncertainty, differential diagnosis, plans for diagnosis and treatment, estimated length of stay (LOS), potential for upgrade in status to a higher level of care, and consideration of disposition. Although the final three items are not core clinical reasoning domains in the medical education literature, they represent clinical judgments that are especially relevant for the delivery of the high-quality and cost-effective care of hospitalized patients. Given that the probabilities and estimations of these three elements evolve over the course of any hospitalization on the basis of test results and response to therapy, the documentation of initial expectations on these fronts can facilitate distributed cognition with all individuals becoming wiser from shared insights.¹⁰ The tool uses two- and three-point rating scales, with each number score being clearly defined by specific written criteria (total score range: 0-14; Appendix).

Data Collection

Hospitalists’ admission notes from the three hospitals were used to validate the CRANAPL tool. Admission notes from patients hospitalized to the general medical floors with an admission diagnosis of either fever, syncope/dizziness, or abdominal pain were used. These diagnoses were purposefully examined because they (1) have a wide differential diagnosis, (2) are common presenting symptoms, and (3) are prone to diagnostic errors.^29-32

The centralized EHR system across the three hospitals identified admission notes with one of these primary diagnoses of patients admitted over the period of January 2014 to October 2017. We submitted a request for 650 admission notes to be randomly selected from the centralized institutional records system. The notes were stratified by hospital and diagnosis. The sample size of our study was comparable with that of prior psychometric validation studies.^33,34 Upon reviewing the A&Ps associated with these admissions, 365 notes were excluded for one of three reasons: (1) the note was written by a nurse practitioner, physician assistant, resident, or medical student; (2) the admission diagnosis had been definitively confirmed in the emergency department (eg, abdominal pain due to diverticulitis seen on CT); and (3) the note represented the fourth or more note by any single provider (to sample notes of many providers, no more than three notes written by any single provider were analyzed). A total of 285 admission notes were ultimately included in the sample.

Data were deidentified, and the A&P sections of the admission notes were each copied from the EHR into a unique Word document. Patient and hospital demographic data (including age, gender, race, number of comorbid conditions, LOS, hospital charges, and readmission to the same health system within 30 days) were collected separately from the EHR. Select physician characteristics were also collected from the hospitalist groups at each of the three hospitals, as was the length (word count) of each A&P.

The study was approved by our institutional review board.

Data Analysis

Two authors scored all deidentified A&Ps by using the finalized version of the CRANAPL tool. Prior to using the CRANAPL tool on each of the notes, these raters read each A&P and scored them by using two single-item rating scales: a global clinical reasoning and a global readability/clarity measure. Both of these global scales used three-item Likert scales (below average, average, and above average). These global rating scales collected the reviewers’ gestalt about the quality and clarity of the A&P. The use of gestalt ratings as comparators is supported by other research.³⁵

Descriptive statistics were computed for all variables. Each rater rescored a sample of 48 records (one month after the initial scoring) and intraclass correlations (ICCs) were computed for intrarater reliability. ICCs were calculated for each item and for the CRANAPL total to determine interrater reliability.

The averaged ratings from the two raters were used for all other analyses. For CRANAPL’s internal structure validity evidence, Cronbach’s alpha was calculated as a measure of internal consistency. For relations to other variables validity evidence, CRANAPL total scores were compared with the two global assessment variables with linear regressions.

Bivariate analyses were performed by applying parametric and nonparametric tests as appropriate. A series of multivariate linear regressions, controlling for diagnosis and clustered variance by hospital site, were performed using CRANAPL total as the dependent variable and patient variables as predictors.

All data were analyzed using Stata (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, Texas: StataCorp LP.)

The admission notes of 120 hospitalists were evaluated (Table 1). A total of 39 (33%) physicians were moonlighters with primary appointments outside of the hospitalist division, and 81 (68%) were full-time hospitalists. Among the 120 hospitalists, 48 (40%) were female, 60 (50%) were international medical graduates, and 90 (75%) were of nonwhite race. Most hospitalist physicians (n = 47, 58%) had worked in our health system for less than five years, and 64 hospitalists (53%) devoted greater than 50% of their time to patient care.

Approximately equal numbers of patient admission notes were pulled from each of the three hospitals. The average age of patients was 67.2 (SD 13.6) years, 145 (51%) were female, and 120 (42%) were of nonwhite race. The mean LOS for all patients was 4.0 (SD 3.4) days. A total of 44 (15%) patients were readmitted to the same health system within 30 days of discharge. None of the patients died during the incident hospitalization. The average charge for each of the hospitalizations was $10,646 (SD $9,964).

CRANAPL Data

Figure 1 shows the distribution of the scores given by each rater for each of the nine items. The mean of the total CRANAPL score given by both raters was 6.4 (SD 2.2). Scoring for some items were high (eg, summary statement: 1.5/2), whereas performance on others were low (eg, estimating LOS: 0.1/1 and describing the potential need for upgrade in care: 0.0/1).

Validity of the CRANAPL Tool’s Internal Structure

Cronbach’s alpha, which was used to measure internal consistency within the CRANAPL tool, was 0.43. The ICC, which was applied to measure the interrater reliability for both raters for the total CRANAPL score, was 0.83 (95% CI:  0.76-0.87). The ICC values for intrarater reliability for raters 1 and 2 were 0.73 (95% CI: 0.60-0.83) and 0.73 (95% CI: 0.45-0.86), respectively.

Relations to Other Variables Validity

Associations between CRANAPL total scores, global clinical reasoning, and global scores for note readability/clarity were statistically significant (P < .001), Figure 2.

Eight out of nine CRANAPL variables were statistically significantly different across the three hospitals (P <. 01) when data were analyzed by hospital site. Hospital C had the highest mean score of 7.4 (SD 2.0), followed by Hospital B with a score of 6.6 (SD 2.1), and Hospital A had the lowest total CRANAPL score of 5.2 (SD 1.9). This difference was statistically significant (P < .001). Five variables with respect to admission diagnoses (uncertainty acknowledged, differential diagnosis, plan for diagnosis, plan for treatment, and upgrade plan) were statistically significantly different across notes. Notes for syncope/dizziness generally yielded higher scores than those for abdominal pain and fever.

Factors Associated with High CRANAPL Scores

Table 2 shows the associations between CRANAPL scores and several covariates. Before adjustment, high CRANAPL scores were associated with high word counts of A&Ps (P < .001) and high hospital charges (P < .05). These associations were no longer significant after adjusting for hospital site and admitting diagnoses.

We reviewed the documentation of clinical reasoning in 285 admission notes at three different hospitals written by hospitalist physicians during routine clinical care. To our knowledge, this is the first study that assessed the documentation of hospitalists’ clinical reasoning with real patient notes. Wide variability exists in the documentation of clinical reasoning within the A&Ps of hospitalists’ admission notes. We have provided validity evidence to support the use of the user-friendly CRANAPL tool.

Prior studies have described rubrics for evaluating the clinical reasoning skills of medical students.^14,15 The ICCs for the IDEA rubric used to assess medical students’ documentation of clinical reasoning were fair to moderate (0.29-0.67), whereas the ICC for the CRANAPL tool was high at 0.83. This measure of reliability is similar to that for the P-HAPEE rubric used to assess medical students’ documentation of pediatric history and physical notes.¹⁵ These data are markedly different from the data in previous studies that have found low interrater reliability for psychometric evaluations related to judgment and decision-making.^36-39 CRANAPL was also found to have high intrarater reliability, which shows the reproducibility of an individual’s assessment over time. The strong association between the total CRANAPL score and global clinical reasoning assessment found in the present study is similar to that found in previous studies that have also embedded global rating scales as comparators when assessing clinical reasoning.^13,,15,40,41 Global rating scales represent an overarching structure for comparison given the absence of an accepted method or gold standard for assessing clinical reasoning documentation. High-quality provider notes are defined by clarity, thoroughness, and accuracy;³⁵ and effective documentation promotes communication and the coordination of care among the members of the care team.³

The total CRANAPL scores varied by hospital site with academic hospitals (B and C) scoring higher than the community hospital (A) in our study. Similarly, lengthy A&Ps were associated with high CRANAPL scores (P < .001) prior to adjustment for hospital site. Healthcare providers consider that the thoroughness of documentation denotes quality and attention to detail.^35,42 Comprehensive documentation takes time; the longer notes by academic hospitalists than those by community hospitalists may be attributed to the fewer number of patients generally carried by hospitalists at academic centers than that by hospitalists at community hospitals.⁴³

The documentation of the estimations of LOS, possibility of potential upgrade, and thoughts about disposition were consistently poorly described across all hospital sites and diagnoses. In contrast to CRANAPL, other clinical reasoning rubrics have excluded these items or discussed uncertainty.^14,15,44 These elements represent the forward thinking that may be essential for high-quality progressive care by hospitalists. Physicians’s difficulty in acknowledging uncertainty has been associated with resource overuse, including the excessive ordering of tests, iatrogenic injury, and heavy financial burden on the healthcare system.^45,46 The lack of thoughtful clinical and management reasoning at the time of admission is believed to be associated with medical errors.⁴⁷ If used as a guide, the CRANAPL tool may promote reflection on the part of the admitting physician. The estimations of LOS, potential for upgrade to a higher level of care, and disposition are markers of optimal inpatient care, especially for hospitalists who work in shifts with embedded handoffs. When shared with colleagues (through documentation), there is the potential for distributed cognition¹⁰ to extend throughout the social network of the hospitalist group. The fact that so few providers are currently including these items in their A&P’s show that the providers are either not performing or documenting the ‘reasoning’. Either way, this is an opportunity that has been highlighted by the CRANAPL tool.

Several limitations of this study should be considered. First, the CRANAPL tool may not have captured elements of optimal clinical reasoning documentation. The reliance on multiple methods and an iterative process in the refinement of the CRANAPL tool should have minimized this. Second, this study was conducted across a single healthcare system that uses the same EHR; this EHR or institutional culture may influence documentation practices and behaviors. Given that using the CRANAPL tool to score an A&P is quick and easy, the benefit of giving providers feedback on their notes remains to be seen—here and at other hospitals. Third, our sample size could limit the generalizability of the results and the significance of the associations. However, the sample assessed in our study was significantly larger than that assessed in other studies that have validated clinical reasoning rubrics.^14,15 Fourth, clinical reasoning is a broad and multidimensional construct. The CRANAPL tool focuses exclusively on hospitalists’ documentation of clinical reasoning and therefore does not assess aspects of clinical reasoning occurring in the physicians’ minds. Finally, given our goal to optimally validate the CRANAPL tool, we chose to test the tool on specific presentations that are known to be associated with diagnostic practice variation and errors. We may have observed different results had we chosen a different set of diagnoses from each hospital. Further validity evidence will be established when applying the CRANPL tool to different diagnoses and to notes from other clinical settings.

In conclusion, this study focuses on the development and validation of the CRANAPL tool that assesses how hospitalists document their clinical reasoning in the A&P section of admission notes. Our results show that wide variability exists in the documentation of clinical reasoning by hospitalists within and across hospitals. Given the CRANAPL tool’s ease-of-use and its versatility, hospitalist divisions in academic and nonacademic settings may use the CRANAPL tool to assess and provide feedback on the documentation of hospitalists’ clinical reasoning. Beyond studying whether physicians can be taught to improve their notes with feedback based on the CRANAPL tool, future studies may explore whether enhancing clinical reasoning documentation may be associated with improvements in patient care and clinical outcomes.

Acknowledgments

Dr. Wright is the Anne Gaines and G. Thomas Miller Professor of Medicine which is supported through Hopkins’ Center for Innovative Medicine.

The authors thank Christine Caufield-Noll, MLIS, AHIP (Johns Hopkins Bayview Medical Center, Baltimore, Maryland) for her assistance with this project.

Disclosures

The authors have nothing to disclose.

Approximately 60,000 hospitalists were working in the United States in 2018.¹ Hospitalist groups work collaboratively because of the shiftwork required for 24/7 patient coverage, and first-rate clinical documentation is essential for quality care.² Thoughtful clinical documentation not only transmits one provider’s clinical reasoning to other providers but is a professional responsibility.³ Hospitalists spend two-thirds of their time in indirect patient-care activities and approximately one quarter of their time on documentation in electronic health records (EHRs).⁴ Despite documentation occupying a substantial portion of the clinician’s time, published literature on the best practices for the documentation of clinical reasoning in hospital medicine or its assessment remains scant.^5-7

Clinical reasoning involves establishing a diagnosis and developing a therapeutic plan that fits the unique circumstances and needs of the patient.⁸ Inpatient providers who admit patients to the hospital end the admission note with their assessment and plan (A&P) after reflecting about a patient’s presenting illness. The A&P generally represents the interpretations, deductions, and clinical reasoning of the inpatient providers; this is the section of the note that fellow physicians concentrate on over others.⁹ The documentation of clinical reasoning in the A&P allows for many to consider how the recorded interpretations relate to their own elucidations resulting in distributed cognition.¹⁰

Disorganized documentation can contribute to cognitive overload and impede thoughtful consideration about the clinical presentation.³ The assessment of clinical documentation may translate into reduced medical errors and improved note quality.^11,12 Studies that have formally evaluated the documentation of clinical reasoning have focused exclusively on medical students.^13-15 The nonexistence of a detailed rubric for evaluating clinical reasoning in the A&Ps of hospitalists represents a missed opportunity for evaluating what hospitalists “do”; if this evolves into a mechanism for offering formative feedback, such professional development would impact the highest level of Miller’s assessment pyramid.¹⁶ We therefore undertook this study to establish a metric to assess the hospitalist providers’ documentation of clinical reasoning in the A&P of an admission note.

Study Design, Setting, and Subjects

This was a retrospective study that reviewed the admission notes of hospitalists for patients admitted over the period of January 2014 and October 2017 at three hospitals in Maryland. One is a community hospital (Hospital A) and two are academic medical centers (Hospital B and Hospital C). Even though these three hospitals are part of one health system, they have distinct cultures and leadership, serve different populations, and are staffed by different provider teams.

The notes of physicians working for the hospitalist groups at each of the three hospitals were the focus of the analysis in this study.

Development of the Documentation Assessment Rubric

A team was assembled to develop the Clinical Reasoning in Admission Note Assessment & PLan (CRANAPL) tool. The CRANAPL was designed to assess the comprehensiveness and thoughtfulness of the clinical reasoning documented in the A&P sections of the notes of patients who were admitted to the hospital with an acute illness. Validity evidence for CRANAPL was summarized on the basis of Messick’s unified validity framework by using four of the five sources of validity: content, response process, internal structure, and relations to other variables.¹⁷

Content Validity

The development team consisted of members who have an average of 10 years of clinical experience in hospital medicine; have studied clinical excellence and clinical reasoning; and have expertise in feedback, assessment, and professional development.^18-22 The development of the CRANAPL tool by the team was informed by a review of the clinical reasoning literature, with particular attention paid to the standards and competencies outlined by the Liaison Committee on Medical Education, the Association of American Medical Colleges, the Accreditation Council on Graduate Medical Education, the Internal Medicine Milestone Project, and the Society of Hospital Medicine.^23-26 For each of these parties, diagnostic reasoning and its impact on clinical decision-making are considered to be a core competency. Several works that heavily influenced the CRANAPL tool’s development were Baker’s Interpretive Summary, Differential Diagnosis, Explanation of Reasoning, And Alternatives (IDEA) assessment tool;¹⁴ King’s Pediatric History and Physical Exam Evaluation (P-HAPEE) rubric;¹⁵ and three other studies related to diagnostic reasoning.^16,27,28 These manuscripts and other works substantively informed the preliminary behavioral-based anchors that formed the initial foundation for the tool under development. The CRANAPL tool was shown to colleagues at other institutions who are leaders on clinical reasoning and was presented at academic conferences in the Division of General Internal Medicine and the Division of Hospital Medicine of our institution. Feedback resulted in iterative revisions. The aforementioned methods established content validity evidence for the CRANAPL tool.

Response Process Validity

Several of the authors pilot-tested earlier iterations on admission notes that were excluded from the sample when refining the CRANAPL tool. The weaknesses and sources of confusion with specific items were addressed by scoring 10 A&Ps individually and then comparing data captured on the tool. This cycle was repeated three times for the iterative enhancement and finalization of the CRANAPL tool. On several occasions when two authors were piloting the near-final CRANAPL tool, a third author interviewed each of the two authors about reactivity while assessing individual items and exploring with probes how their own clinical documentation practices were being considered when scoring the notes. The reasonable and thoughtful answers provided by the two authors as they explained and justified the scores they were selecting during the pilot testing served to confer response process validity evidence.

Finalizing the CRANAPL Tool

The nine-item CRANAPL tool includes elements for problem representation, leading diagnosis, uncertainty, differential diagnosis, plans for diagnosis and treatment, estimated length of stay (LOS), potential for upgrade in status to a higher level of care, and consideration of disposition. Although the final three items are not core clinical reasoning domains in the medical education literature, they represent clinical judgments that are especially relevant for the delivery of the high-quality and cost-effective care of hospitalized patients. Given that the probabilities and estimations of these three elements evolve over the course of any hospitalization on the basis of test results and response to therapy, the documentation of initial expectations on these fronts can facilitate distributed cognition with all individuals becoming wiser from shared insights.¹⁰ The tool uses two- and three-point rating scales, with each number score being clearly defined by specific written criteria (total score range: 0-14; Appendix).

Data Collection

Hospitalists’ admission notes from the three hospitals were used to validate the CRANAPL tool. Admission notes from patients hospitalized to the general medical floors with an admission diagnosis of either fever, syncope/dizziness, or abdominal pain were used. These diagnoses were purposefully examined because they (1) have a wide differential diagnosis, (2) are common presenting symptoms, and (3) are prone to diagnostic errors.^29-32

The centralized EHR system across the three hospitals identified admission notes with one of these primary diagnoses of patients admitted over the period of January 2014 to October 2017. We submitted a request for 650 admission notes to be randomly selected from the centralized institutional records system. The notes were stratified by hospital and diagnosis. The sample size of our study was comparable with that of prior psychometric validation studies.^33,34 Upon reviewing the A&Ps associated with these admissions, 365 notes were excluded for one of three reasons: (1) the note was written by a nurse practitioner, physician assistant, resident, or medical student; (2) the admission diagnosis had been definitively confirmed in the emergency department (eg, abdominal pain due to diverticulitis seen on CT); and (3) the note represented the fourth or more note by any single provider (to sample notes of many providers, no more than three notes written by any single provider were analyzed). A total of 285 admission notes were ultimately included in the sample.

Data were deidentified, and the A&P sections of the admission notes were each copied from the EHR into a unique Word document. Patient and hospital demographic data (including age, gender, race, number of comorbid conditions, LOS, hospital charges, and readmission to the same health system within 30 days) were collected separately from the EHR. Select physician characteristics were also collected from the hospitalist groups at each of the three hospitals, as was the length (word count) of each A&P.

The study was approved by our institutional review board.

Data Analysis

Two authors scored all deidentified A&Ps by using the finalized version of the CRANAPL tool. Prior to using the CRANAPL tool on each of the notes, these raters read each A&P and scored them by using two single-item rating scales: a global clinical reasoning and a global readability/clarity measure. Both of these global scales used three-item Likert scales (below average, average, and above average). These global rating scales collected the reviewers’ gestalt about the quality and clarity of the A&P. The use of gestalt ratings as comparators is supported by other research.³⁵

Descriptive statistics were computed for all variables. Each rater rescored a sample of 48 records (one month after the initial scoring) and intraclass correlations (ICCs) were computed for intrarater reliability. ICCs were calculated for each item and for the CRANAPL total to determine interrater reliability.

The averaged ratings from the two raters were used for all other analyses. For CRANAPL’s internal structure validity evidence, Cronbach’s alpha was calculated as a measure of internal consistency. For relations to other variables validity evidence, CRANAPL total scores were compared with the two global assessment variables with linear regressions.

Bivariate analyses were performed by applying parametric and nonparametric tests as appropriate. A series of multivariate linear regressions, controlling for diagnosis and clustered variance by hospital site, were performed using CRANAPL total as the dependent variable and patient variables as predictors.

All data were analyzed using Stata (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, Texas: StataCorp LP.)

The admission notes of 120 hospitalists were evaluated (Table 1). A total of 39 (33%) physicians were moonlighters with primary appointments outside of the hospitalist division, and 81 (68%) were full-time hospitalists. Among the 120 hospitalists, 48 (40%) were female, 60 (50%) were international medical graduates, and 90 (75%) were of nonwhite race. Most hospitalist physicians (n = 47, 58%) had worked in our health system for less than five years, and 64 hospitalists (53%) devoted greater than 50% of their time to patient care.

Approximately equal numbers of patient admission notes were pulled from each of the three hospitals. The average age of patients was 67.2 (SD 13.6) years, 145 (51%) were female, and 120 (42%) were of nonwhite race. The mean LOS for all patients was 4.0 (SD 3.4) days. A total of 44 (15%) patients were readmitted to the same health system within 30 days of discharge. None of the patients died during the incident hospitalization. The average charge for each of the hospitalizations was $10,646 (SD $9,964).

CRANAPL Data

Figure 1 shows the distribution of the scores given by each rater for each of the nine items. The mean of the total CRANAPL score given by both raters was 6.4 (SD 2.2). Scoring for some items were high (eg, summary statement: 1.5/2), whereas performance on others were low (eg, estimating LOS: 0.1/1 and describing the potential need for upgrade in care: 0.0/1).

Validity of the CRANAPL Tool’s Internal Structure

Cronbach’s alpha, which was used to measure internal consistency within the CRANAPL tool, was 0.43. The ICC, which was applied to measure the interrater reliability for both raters for the total CRANAPL score, was 0.83 (95% CI:  0.76-0.87). The ICC values for intrarater reliability for raters 1 and 2 were 0.73 (95% CI: 0.60-0.83) and 0.73 (95% CI: 0.45-0.86), respectively.

Relations to Other Variables Validity

Associations between CRANAPL total scores, global clinical reasoning, and global scores for note readability/clarity were statistically significant (P < .001), Figure 2.

Eight out of nine CRANAPL variables were statistically significantly different across the three hospitals (P <. 01) when data were analyzed by hospital site. Hospital C had the highest mean score of 7.4 (SD 2.0), followed by Hospital B with a score of 6.6 (SD 2.1), and Hospital A had the lowest total CRANAPL score of 5.2 (SD 1.9). This difference was statistically significant (P < .001). Five variables with respect to admission diagnoses (uncertainty acknowledged, differential diagnosis, plan for diagnosis, plan for treatment, and upgrade plan) were statistically significantly different across notes. Notes for syncope/dizziness generally yielded higher scores than those for abdominal pain and fever.

Factors Associated with High CRANAPL Scores

Table 2 shows the associations between CRANAPL scores and several covariates. Before adjustment, high CRANAPL scores were associated with high word counts of A&Ps (P < .001) and high hospital charges (P < .05). These associations were no longer significant after adjusting for hospital site and admitting diagnoses.

We reviewed the documentation of clinical reasoning in 285 admission notes at three different hospitals written by hospitalist physicians during routine clinical care. To our knowledge, this is the first study that assessed the documentation of hospitalists’ clinical reasoning with real patient notes. Wide variability exists in the documentation of clinical reasoning within the A&Ps of hospitalists’ admission notes. We have provided validity evidence to support the use of the user-friendly CRANAPL tool.

Prior studies have described rubrics for evaluating the clinical reasoning skills of medical students.^14,15 The ICCs for the IDEA rubric used to assess medical students’ documentation of clinical reasoning were fair to moderate (0.29-0.67), whereas the ICC for the CRANAPL tool was high at 0.83. This measure of reliability is similar to that for the P-HAPEE rubric used to assess medical students’ documentation of pediatric history and physical notes.¹⁵ These data are markedly different from the data in previous studies that have found low interrater reliability for psychometric evaluations related to judgment and decision-making.^36-39 CRANAPL was also found to have high intrarater reliability, which shows the reproducibility of an individual’s assessment over time. The strong association between the total CRANAPL score and global clinical reasoning assessment found in the present study is similar to that found in previous studies that have also embedded global rating scales as comparators when assessing clinical reasoning.^13,,15,40,41 Global rating scales represent an overarching structure for comparison given the absence of an accepted method or gold standard for assessing clinical reasoning documentation. High-quality provider notes are defined by clarity, thoroughness, and accuracy;³⁵ and effective documentation promotes communication and the coordination of care among the members of the care team.³

The total CRANAPL scores varied by hospital site with academic hospitals (B and C) scoring higher than the community hospital (A) in our study. Similarly, lengthy A&Ps were associated with high CRANAPL scores (P < .001) prior to adjustment for hospital site. Healthcare providers consider that the thoroughness of documentation denotes quality and attention to detail.^35,42 Comprehensive documentation takes time; the longer notes by academic hospitalists than those by community hospitalists may be attributed to the fewer number of patients generally carried by hospitalists at academic centers than that by hospitalists at community hospitals.⁴³

The documentation of the estimations of LOS, possibility of potential upgrade, and thoughts about disposition were consistently poorly described across all hospital sites and diagnoses. In contrast to CRANAPL, other clinical reasoning rubrics have excluded these items or discussed uncertainty.^14,15,44 These elements represent the forward thinking that may be essential for high-quality progressive care by hospitalists. Physicians’s difficulty in acknowledging uncertainty has been associated with resource overuse, including the excessive ordering of tests, iatrogenic injury, and heavy financial burden on the healthcare system.^45,46 The lack of thoughtful clinical and management reasoning at the time of admission is believed to be associated with medical errors.⁴⁷ If used as a guide, the CRANAPL tool may promote reflection on the part of the admitting physician. The estimations of LOS, potential for upgrade to a higher level of care, and disposition are markers of optimal inpatient care, especially for hospitalists who work in shifts with embedded handoffs. When shared with colleagues (through documentation), there is the potential for distributed cognition¹⁰ to extend throughout the social network of the hospitalist group. The fact that so few providers are currently including these items in their A&P’s show that the providers are either not performing or documenting the ‘reasoning’. Either way, this is an opportunity that has been highlighted by the CRANAPL tool.

Several limitations of this study should be considered. First, the CRANAPL tool may not have captured elements of optimal clinical reasoning documentation. The reliance on multiple methods and an iterative process in the refinement of the CRANAPL tool should have minimized this. Second, this study was conducted across a single healthcare system that uses the same EHR; this EHR or institutional culture may influence documentation practices and behaviors. Given that using the CRANAPL tool to score an A&P is quick and easy, the benefit of giving providers feedback on their notes remains to be seen—here and at other hospitals. Third, our sample size could limit the generalizability of the results and the significance of the associations. However, the sample assessed in our study was significantly larger than that assessed in other studies that have validated clinical reasoning rubrics.^14,15 Fourth, clinical reasoning is a broad and multidimensional construct. The CRANAPL tool focuses exclusively on hospitalists’ documentation of clinical reasoning and therefore does not assess aspects of clinical reasoning occurring in the physicians’ minds. Finally, given our goal to optimally validate the CRANAPL tool, we chose to test the tool on specific presentations that are known to be associated with diagnostic practice variation and errors. We may have observed different results had we chosen a different set of diagnoses from each hospital. Further validity evidence will be established when applying the CRANPL tool to different diagnoses and to notes from other clinical settings.

In conclusion, this study focuses on the development and validation of the CRANAPL tool that assesses how hospitalists document their clinical reasoning in the A&P section of admission notes. Our results show that wide variability exists in the documentation of clinical reasoning by hospitalists within and across hospitals. Given the CRANAPL tool’s ease-of-use and its versatility, hospitalist divisions in academic and nonacademic settings may use the CRANAPL tool to assess and provide feedback on the documentation of hospitalists’ clinical reasoning. Beyond studying whether physicians can be taught to improve their notes with feedback based on the CRANAPL tool, future studies may explore whether enhancing clinical reasoning documentation may be associated with improvements in patient care and clinical outcomes.

Acknowledgments

Dr. Wright is the Anne Gaines and G. Thomas Miller Professor of Medicine which is supported through Hopkins’ Center for Innovative Medicine.

The authors thank Christine Caufield-Noll, MLIS, AHIP (Johns Hopkins Bayview Medical Center, Baltimore, Maryland) for her assistance with this project.

Disclosures

The authors have nothing to disclose.

Approximately 60,000 hospitalists were working in the United States in 2018.¹ Hospitalist groups work collaboratively because of the shiftwork required for 24/7 patient coverage, and first-rate clinical documentation is essential for quality care.² Thoughtful clinical documentation not only transmits one provider’s clinical reasoning to other providers but is a professional responsibility.³ Hospitalists spend two-thirds of their time in indirect patient-care activities and approximately one quarter of their time on documentation in electronic health records (EHRs).⁴ Despite documentation occupying a substantial portion of the clinician’s time, published literature on the best practices for the documentation of clinical reasoning in hospital medicine or its assessment remains scant.^5-7

Clinical reasoning involves establishing a diagnosis and developing a therapeutic plan that fits the unique circumstances and needs of the patient.⁸ Inpatient providers who admit patients to the hospital end the admission note with their assessment and plan (A&P) after reflecting about a patient’s presenting illness. The A&P generally represents the interpretations, deductions, and clinical reasoning of the inpatient providers; this is the section of the note that fellow physicians concentrate on over others.⁹ The documentation of clinical reasoning in the A&P allows for many to consider how the recorded interpretations relate to their own elucidations resulting in distributed cognition.¹⁰

Disorganized documentation can contribute to cognitive overload and impede thoughtful consideration about the clinical presentation.³ The assessment of clinical documentation may translate into reduced medical errors and improved note quality.^11,12 Studies that have formally evaluated the documentation of clinical reasoning have focused exclusively on medical students.^13-15 The nonexistence of a detailed rubric for evaluating clinical reasoning in the A&Ps of hospitalists represents a missed opportunity for evaluating what hospitalists “do”; if this evolves into a mechanism for offering formative feedback, such professional development would impact the highest level of Miller’s assessment pyramid.¹⁶ We therefore undertook this study to establish a metric to assess the hospitalist providers’ documentation of clinical reasoning in the A&P of an admission note.

Study Design, Setting, and Subjects

This was a retrospective study that reviewed the admission notes of hospitalists for patients admitted over the period of January 2014 and October 2017 at three hospitals in Maryland. One is a community hospital (Hospital A) and two are academic medical centers (Hospital B and Hospital C). Even though these three hospitals are part of one health system, they have distinct cultures and leadership, serve different populations, and are staffed by different provider teams.

The notes of physicians working for the hospitalist groups at each of the three hospitals were the focus of the analysis in this study.

Development of the Documentation Assessment Rubric

A team was assembled to develop the Clinical Reasoning in Admission Note Assessment & PLan (CRANAPL) tool. The CRANAPL was designed to assess the comprehensiveness and thoughtfulness of the clinical reasoning documented in the A&P sections of the notes of patients who were admitted to the hospital with an acute illness. Validity evidence for CRANAPL was summarized on the basis of Messick’s unified validity framework by using four of the five sources of validity: content, response process, internal structure, and relations to other variables.¹⁷

Content Validity

The development team consisted of members who have an average of 10 years of clinical experience in hospital medicine; have studied clinical excellence and clinical reasoning; and have expertise in feedback, assessment, and professional development.^18-22 The development of the CRANAPL tool by the team was informed by a review of the clinical reasoning literature, with particular attention paid to the standards and competencies outlined by the Liaison Committee on Medical Education, the Association of American Medical Colleges, the Accreditation Council on Graduate Medical Education, the Internal Medicine Milestone Project, and the Society of Hospital Medicine.^23-26 For each of these parties, diagnostic reasoning and its impact on clinical decision-making are considered to be a core competency. Several works that heavily influenced the CRANAPL tool’s development were Baker’s Interpretive Summary, Differential Diagnosis, Explanation of Reasoning, And Alternatives (IDEA) assessment tool;¹⁴ King’s Pediatric History and Physical Exam Evaluation (P-HAPEE) rubric;¹⁵ and three other studies related to diagnostic reasoning.^16,27,28 These manuscripts and other works substantively informed the preliminary behavioral-based anchors that formed the initial foundation for the tool under development. The CRANAPL tool was shown to colleagues at other institutions who are leaders on clinical reasoning and was presented at academic conferences in the Division of General Internal Medicine and the Division of Hospital Medicine of our institution. Feedback resulted in iterative revisions. The aforementioned methods established content validity evidence for the CRANAPL tool.

Response Process Validity

Several of the authors pilot-tested earlier iterations on admission notes that were excluded from the sample when refining the CRANAPL tool. The weaknesses and sources of confusion with specific items were addressed by scoring 10 A&Ps individually and then comparing data captured on the tool. This cycle was repeated three times for the iterative enhancement and finalization of the CRANAPL tool. On several occasions when two authors were piloting the near-final CRANAPL tool, a third author interviewed each of the two authors about reactivity while assessing individual items and exploring with probes how their own clinical documentation practices were being considered when scoring the notes. The reasonable and thoughtful answers provided by the two authors as they explained and justified the scores they were selecting during the pilot testing served to confer response process validity evidence.

Finalizing the CRANAPL Tool

The nine-item CRANAPL tool includes elements for problem representation, leading diagnosis, uncertainty, differential diagnosis, plans for diagnosis and treatment, estimated length of stay (LOS), potential for upgrade in status to a higher level of care, and consideration of disposition. Although the final three items are not core clinical reasoning domains in the medical education literature, they represent clinical judgments that are especially relevant for the delivery of the high-quality and cost-effective care of hospitalized patients. Given that the probabilities and estimations of these three elements evolve over the course of any hospitalization on the basis of test results and response to therapy, the documentation of initial expectations on these fronts can facilitate distributed cognition with all individuals becoming wiser from shared insights.¹⁰ The tool uses two- and three-point rating scales, with each number score being clearly defined by specific written criteria (total score range: 0-14; Appendix).

Data Collection

Hospitalists’ admission notes from the three hospitals were used to validate the CRANAPL tool. Admission notes from patients hospitalized to the general medical floors with an admission diagnosis of either fever, syncope/dizziness, or abdominal pain were used. These diagnoses were purposefully examined because they (1) have a wide differential diagnosis, (2) are common presenting symptoms, and (3) are prone to diagnostic errors.^29-32

The centralized EHR system across the three hospitals identified admission notes with one of these primary diagnoses of patients admitted over the period of January 2014 to October 2017. We submitted a request for 650 admission notes to be randomly selected from the centralized institutional records system. The notes were stratified by hospital and diagnosis. The sample size of our study was comparable with that of prior psychometric validation studies.^33,34 Upon reviewing the A&Ps associated with these admissions, 365 notes were excluded for one of three reasons: (1) the note was written by a nurse practitioner, physician assistant, resident, or medical student; (2) the admission diagnosis had been definitively confirmed in the emergency department (eg, abdominal pain due to diverticulitis seen on CT); and (3) the note represented the fourth or more note by any single provider (to sample notes of many providers, no more than three notes written by any single provider were analyzed). A total of 285 admission notes were ultimately included in the sample.

Data were deidentified, and the A&P sections of the admission notes were each copied from the EHR into a unique Word document. Patient and hospital demographic data (including age, gender, race, number of comorbid conditions, LOS, hospital charges, and readmission to the same health system within 30 days) were collected separately from the EHR. Select physician characteristics were also collected from the hospitalist groups at each of the three hospitals, as was the length (word count) of each A&P.

The study was approved by our institutional review board.

Data Analysis

Two authors scored all deidentified A&Ps by using the finalized version of the CRANAPL tool. Prior to using the CRANAPL tool on each of the notes, these raters read each A&P and scored them by using two single-item rating scales: a global clinical reasoning and a global readability/clarity measure. Both of these global scales used three-item Likert scales (below average, average, and above average). These global rating scales collected the reviewers’ gestalt about the quality and clarity of the A&P. The use of gestalt ratings as comparators is supported by other research.³⁵

Descriptive statistics were computed for all variables. Each rater rescored a sample of 48 records (one month after the initial scoring) and intraclass correlations (ICCs) were computed for intrarater reliability. ICCs were calculated for each item and for the CRANAPL total to determine interrater reliability.

The averaged ratings from the two raters were used for all other analyses. For CRANAPL’s internal structure validity evidence, Cronbach’s alpha was calculated as a measure of internal consistency. For relations to other variables validity evidence, CRANAPL total scores were compared with the two global assessment variables with linear regressions.

Bivariate analyses were performed by applying parametric and nonparametric tests as appropriate. A series of multivariate linear regressions, controlling for diagnosis and clustered variance by hospital site, were performed using CRANAPL total as the dependent variable and patient variables as predictors.

All data were analyzed using Stata (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, Texas: StataCorp LP.)

The admission notes of 120 hospitalists were evaluated (Table 1). A total of 39 (33%) physicians were moonlighters with primary appointments outside of the hospitalist division, and 81 (68%) were full-time hospitalists. Among the 120 hospitalists, 48 (40%) were female, 60 (50%) were international medical graduates, and 90 (75%) were of nonwhite race. Most hospitalist physicians (n = 47, 58%) had worked in our health system for less than five years, and 64 hospitalists (53%) devoted greater than 50% of their time to patient care.

Approximately equal numbers of patient admission notes were pulled from each of the three hospitals. The average age of patients was 67.2 (SD 13.6) years, 145 (51%) were female, and 120 (42%) were of nonwhite race. The mean LOS for all patients was 4.0 (SD 3.4) days. A total of 44 (15%) patients were readmitted to the same health system within 30 days of discharge. None of the patients died during the incident hospitalization. The average charge for each of the hospitalizations was $10,646 (SD $9,964).

CRANAPL Data

Figure 1 shows the distribution of the scores given by each rater for each of the nine items. The mean of the total CRANAPL score given by both raters was 6.4 (SD 2.2). Scoring for some items were high (eg, summary statement: 1.5/2), whereas performance on others were low (eg, estimating LOS: 0.1/1 and describing the potential need for upgrade in care: 0.0/1).

Validity of the CRANAPL Tool’s Internal Structure

Cronbach’s alpha, which was used to measure internal consistency within the CRANAPL tool, was 0.43. The ICC, which was applied to measure the interrater reliability for both raters for the total CRANAPL score, was 0.83 (95% CI:  0.76-0.87). The ICC values for intrarater reliability for raters 1 and 2 were 0.73 (95% CI: 0.60-0.83) and 0.73 (95% CI: 0.45-0.86), respectively.

Relations to Other Variables Validity

Associations between CRANAPL total scores, global clinical reasoning, and global scores for note readability/clarity were statistically significant (P < .001), Figure 2.

Eight out of nine CRANAPL variables were statistically significantly different across the three hospitals (P <. 01) when data were analyzed by hospital site. Hospital C had the highest mean score of 7.4 (SD 2.0), followed by Hospital B with a score of 6.6 (SD 2.1), and Hospital A had the lowest total CRANAPL score of 5.2 (SD 1.9). This difference was statistically significant (P < .001). Five variables with respect to admission diagnoses (uncertainty acknowledged, differential diagnosis, plan for diagnosis, plan for treatment, and upgrade plan) were statistically significantly different across notes. Notes for syncope/dizziness generally yielded higher scores than those for abdominal pain and fever.

Factors Associated with High CRANAPL Scores

Table 2 shows the associations between CRANAPL scores and several covariates. Before adjustment, high CRANAPL scores were associated with high word counts of A&Ps (P < .001) and high hospital charges (P < .05). These associations were no longer significant after adjusting for hospital site and admitting diagnoses.

We reviewed the documentation of clinical reasoning in 285 admission notes at three different hospitals written by hospitalist physicians during routine clinical care. To our knowledge, this is the first study that assessed the documentation of hospitalists’ clinical reasoning with real patient notes. Wide variability exists in the documentation of clinical reasoning within the A&Ps of hospitalists’ admission notes. We have provided validity evidence to support the use of the user-friendly CRANAPL tool.

Prior studies have described rubrics for evaluating the clinical reasoning skills of medical students.^14,15 The ICCs for the IDEA rubric used to assess medical students’ documentation of clinical reasoning were fair to moderate (0.29-0.67), whereas the ICC for the CRANAPL tool was high at 0.83. This measure of reliability is similar to that for the P-HAPEE rubric used to assess medical students’ documentation of pediatric history and physical notes.¹⁵ These data are markedly different from the data in previous studies that have found low interrater reliability for psychometric evaluations related to judgment and decision-making.^36-39 CRANAPL was also found to have high intrarater reliability, which shows the reproducibility of an individual’s assessment over time. The strong association between the total CRANAPL score and global clinical reasoning assessment found in the present study is similar to that found in previous studies that have also embedded global rating scales as comparators when assessing clinical reasoning.^13,,15,40,41 Global rating scales represent an overarching structure for comparison given the absence of an accepted method or gold standard for assessing clinical reasoning documentation. High-quality provider notes are defined by clarity, thoroughness, and accuracy;³⁵ and effective documentation promotes communication and the coordination of care among the members of the care team.³

The total CRANAPL scores varied by hospital site with academic hospitals (B and C) scoring higher than the community hospital (A) in our study. Similarly, lengthy A&Ps were associated with high CRANAPL scores (P < .001) prior to adjustment for hospital site. Healthcare providers consider that the thoroughness of documentation denotes quality and attention to detail.^35,42 Comprehensive documentation takes time; the longer notes by academic hospitalists than those by community hospitalists may be attributed to the fewer number of patients generally carried by hospitalists at academic centers than that by hospitalists at community hospitals.⁴³

The documentation of the estimations of LOS, possibility of potential upgrade, and thoughts about disposition were consistently poorly described across all hospital sites and diagnoses. In contrast to CRANAPL, other clinical reasoning rubrics have excluded these items or discussed uncertainty.^14,15,44 These elements represent the forward thinking that may be essential for high-quality progressive care by hospitalists. Physicians’s difficulty in acknowledging uncertainty has been associated with resource overuse, including the excessive ordering of tests, iatrogenic injury, and heavy financial burden on the healthcare system.^45,46 The lack of thoughtful clinical and management reasoning at the time of admission is believed to be associated with medical errors.⁴⁷ If used as a guide, the CRANAPL tool may promote reflection on the part of the admitting physician. The estimations of LOS, potential for upgrade to a higher level of care, and disposition are markers of optimal inpatient care, especially for hospitalists who work in shifts with embedded handoffs. When shared with colleagues (through documentation), there is the potential for distributed cognition¹⁰ to extend throughout the social network of the hospitalist group. The fact that so few providers are currently including these items in their A&P’s show that the providers are either not performing or documenting the ‘reasoning’. Either way, this is an opportunity that has been highlighted by the CRANAPL tool.

Several limitations of this study should be considered. First, the CRANAPL tool may not have captured elements of optimal clinical reasoning documentation. The reliance on multiple methods and an iterative process in the refinement of the CRANAPL tool should have minimized this. Second, this study was conducted across a single healthcare system that uses the same EHR; this EHR or institutional culture may influence documentation practices and behaviors. Given that using the CRANAPL tool to score an A&P is quick and easy, the benefit of giving providers feedback on their notes remains to be seen—here and at other hospitals. Third, our sample size could limit the generalizability of the results and the significance of the associations. However, the sample assessed in our study was significantly larger than that assessed in other studies that have validated clinical reasoning rubrics.^14,15 Fourth, clinical reasoning is a broad and multidimensional construct. The CRANAPL tool focuses exclusively on hospitalists’ documentation of clinical reasoning and therefore does not assess aspects of clinical reasoning occurring in the physicians’ minds. Finally, given our goal to optimally validate the CRANAPL tool, we chose to test the tool on specific presentations that are known to be associated with diagnostic practice variation and errors. We may have observed different results had we chosen a different set of diagnoses from each hospital. Further validity evidence will be established when applying the CRANPL tool to different diagnoses and to notes from other clinical settings.

In conclusion, this study focuses on the development and validation of the CRANAPL tool that assesses how hospitalists document their clinical reasoning in the A&P section of admission notes. Our results show that wide variability exists in the documentation of clinical reasoning by hospitalists within and across hospitals. Given the CRANAPL tool’s ease-of-use and its versatility, hospitalist divisions in academic and nonacademic settings may use the CRANAPL tool to assess and provide feedback on the documentation of hospitalists’ clinical reasoning. Beyond studying whether physicians can be taught to improve their notes with feedback based on the CRANAPL tool, future studies may explore whether enhancing clinical reasoning documentation may be associated with improvements in patient care and clinical outcomes.

Acknowledgments

Dr. Wright is the Anne Gaines and G. Thomas Miller Professor of Medicine which is supported through Hopkins’ Center for Innovative Medicine.

The authors thank Christine Caufield-Noll, MLIS, AHIP (Johns Hopkins Bayview Medical Center, Baltimore, Maryland) for her assistance with this project.

Disclosures

The authors have nothing to disclose.

A 56-year-old Japanese man with a history of renal transplantation 20 years prior presented to the emergency department (ED) with two months of dyspnea on exertion and one day of fever and chills. The patient was in his usual state of health until two months prior to presentation, when he gradually noticed shortness of breath after sustained or effortful physical activities. The dyspnea improved with rest. Over the following two months, he noticed that the shortness of breath came on with lesser degrees of exertion, such as walking 100 meters. One day before presentation, he developed a fever of 39°C and chills at home, which prompted him to seek ED care. He denied chest pain, cough, leg swelling, or paroxysmal nocturnal dyspnea.

The differential diagnosis of exertional dyspnea progressing over several months includes cardiac, pulmonary, hematologic, and neuromuscular conditions. The patient’s history of renal transplantation prompts consideration of worsening indolent pneumonia (eg, Aspergillus, cytomegalovirus [CMV], or Pneumocystis pneumonia), allograft dysfunction with volume overload, recrudescence of the underlying disease that incited renal failure earlier in life (eg, vasculitis), or a late-onset posttransplantation lymphoproliferative disorder (PTLD). Additionally, acute fever in an immunocompromised patient immediately raises suspicion for infection (eg, pneumonia, enteritis, or urinary tract infection). At this point, it is difficult to know whether the subacute-to-chronic exertional dyspnea and the acute fever are consequences of the same disease or separate, potentially overlapping, processes.

His past medical history was significant for end-stage renal disease due to membranoproliferative glomerular nephropathy (MPGN), for which living, related-donor kidney transplantation was performed 20 years earlier. He also had type 2 diabetes mellitus, hypertension, and basal cell carcinoma of the face, which had been resected three years prior without spread or recurrence. He had no known allergies. Medications included prednisolone 15 mg daily, azathioprine 100 mg daily, and cyclosporine 100 mg daily, as well as amlodipine and candesartan. He lived in Japan with his wife and children. He denied any animal or environmental exposures. He did not smoke cigarettes or drink alcohol and had not traveled recently. His father had diabetes mellitus.

Recrudescence of an underlying autoimmune condition that may have incited MPGN earlier in life is unlikely while taking an immunosuppressive regimen consisting of prednisolone, azathioprine, and cyclosporine. However, these medications do increase susceptibility to infections, lymphoma, and skin cancers. Though he is immunocompromised, the patient is not on prophylaxis for Pneumocystis pneumonia (PCP). PCP in HIV-negative patients is associated with recent glucocorticoid exposure and typically follows an acute-to-subacute course with hypoxemia and respiratory distress. Though the risk of PCP infection is considered highest in the early posttransplantation period (when immunosuppression is most intense), many cases are diagnosed years after transplantation among patients no longer on prophylaxis. The patient has type 2 diabetes mellitus and hypertension, which are known complications of calcineurin inhibitor and steroid therapy and increase the risk of cardiovascular disease. Cardiovascular disease is a major cause of death among renal transplant recipients. Exertional dyspnea may be the presenting symptom of coronary artery disease.

On physical examination, the patient was alert, oriented, and in no acute distress. His temperature was 38.5°C, blood pressure 120/60 mm Hg, heart rate 146 beats per minute, respiratory rate 18 breaths per minute, and oxygen saturation 93% while breathing ambient air. The conjunctiva were normal without pallor or icterus. There was no cervical lymphadenopathy. Cardiac examination revealed tachycardia with a regular rhythm, normal S1 and S2, and no murmurs, rubs, or gallops. Jugular venous pressure was not elevated, and there was no lower extremity edema. Lungs were clear to auscultation bilaterally. The abdomen was soft, nontender, and nondistended. There was no tenderness over the transplanted kidney and no hepatosplenomegaly.

Dyspnea, fever, and tachycardia may be the sole manifestations of pneumonia in solid organ transplant recipients. The absence of cough or adventitious breath sounds does not eliminate concern for pneumonia. Pathogens that cause indolent pneumonia in immunocompromised patients include viruses (such as typical respiratory viruses and CMV), bacteria (typical organisms, Nocardia, Rhodococcus), and mycobacteria. Fungal causes include Aspergillus, Candida, Cryptococcus, Pneumocystis, and endemic mycoses. A detailed environmental history should be taken, and providers should ascertain which fungal diseases are endemic in the patient’s region of residence. There are no examination features suggesting hypervolemia or anemia. Although there is no hepatosplenomegaly or lymphadenopathy, PTLD often involves extranodal tissues, including the lungs. The incidence of PTLD is highest in the 12 months following transplantation, but it may occur at any time in the posttransplantation course. A complete blood count, comprehensive metabolic panel, lactate dehydrogenase (LDH), and blood and sputum cultures are indicated, along with computed tomography (CT) of the chest.

The leukocyte count was 3,500 cells/mm³, the hemoglobin level 9.0 g/dL, mean corpuscular volume 102 fL, and the platelet count 137,000 cells/mm³. The sodium level was 130 mEq/L, potassium 4.6 mEq/L, blood urea nitrogen 41 mg/dL, and creatinine 3.5 mg/dL. These complete blood count and serum electrolyte results were unchanged from the patient’s baseline values. The serum LDH level was 1,895 IU/L (normal range, 115-245 IU/L). The serum ferritin was 2,114 ng/mL (normal range, 13-277 ng/mL). A chest radiograph revealed diffuse, airspace-filling opacities in the bilateral lung bases. The urinalysis was normal. The patient was admitted and started empirically on intravenous ceftriaxone for potential bacterial pneumonia.

Chronic pancytopenia may result from azathioprine or cyclosporine use, marrow suppression or infiltration by a multisystem disease, or nutritional deficiency. Hemophagocytic lymphohistiocytosis (HLH) triggered by infection, a rheumatologic condition, acquired immunodeficiency, or malignancy can present with fevers, pancytopenia, and elevated ferritin, while splenomegaly may be absent. The euvolemic state, baseline creatinine level, and normal urinalysis argue against allograft dysfunction. The elevated serum ferritin nonspecifically confirms systemic inflammation. LDH, an intracellular enzyme involved in the bidirectional conversion of lactate to pyruvate, is expressed across tissue types. Elevated serum LDH attests to cell destruction, in this case potentially from lung infection (such as PCP) or malignancy (such as PTLD). At this point, the differential diagnosis of fever and pulmonary infiltrates in this patient remains broad.

Azathioprine and cyclosporine were stopped. The patient remained febrile despite the administration of intravenous antibiotics. His hypoxia worsened with an oxygen saturation of 90%-93% on 5 L/min of supplemental oxygen administered by nasal cannula. Noncontrast chest CT obtained on the second hospital day revealed ground-glass opacities in the bilateral lung bases. Blood, sputum, and urine cultures were sterile. As empiric therapies, ganciclovir was started for CMV infection, ciprofloxacin added for atypical pneumonia, and trimethoprim-sulfamethoxazole added for Pneumocystis infection.

These chest imaging findings help prioritize the differential diagnosis. Bibasilar ground-glass opacities are evident, while pulmonary masses, nodules, cavitation, adenopathy, and pleural effusions are absent. The differential diagnosis of multifocal ground-glass opacities on chest imaging includes infectious pneumonia, chronic interstitial lung disease, acute alveolar conditions (eg, cardiogenic pulmonary edema, acute respiratory distress syndrome, diffuse alveolar hemorrhage), or other pathologies (eg, drug toxicity, bronchoalveolar carcinoma, cryptogenic organizing pneumonia).

Infectious pneumonia is the principal concern. A diagnosis of PCP could be unifying, given dyspnea, progressive respiratory failure with hypoxia, and elevated LDH in an immunocompromised patient who is not prescribed PCP prophylaxis. The bilateral lung infiltrates and the absence of thoracic adenopathy or pleural effusions are characteristic of PCP as well. However, caution should be exercised in making specific infectious diagnoses in immunocompromised hosts on the basis of clinical and imaging findings alone. There can be overlap in the radiologic appearance of various infections (eg, CMV pneumonia can also present with bilateral ground-glass infiltrates, with concurrent fever, hypoxia, and pancytopenia). Additionally, more than one pneumonic pathogen may be implicated (eg, acute viral pneumonia superimposed on indolent fungal pneumonia). Polymerase chain reaction (PCR) analysis of respiratory secretions for viruses, serum PCR and serologic testing for herpes viruses, and serum beta-D-glucan and galactomannan assays are indicated. Serum serologic testing for fungi and bacteria such as Nocardia can be helpful, though the negative predictive values of these tests may be reduced in patients with impaired humoral immunity. Timely bronchoalveolar lavage (BAL) with microbiologic and PCR analysis and cytology is advised.

Fever, elevated LDH, cytopenias, and pulmonary infiltrates also raise suspicion for an underlying hematologic malignancy, such as PTLD. However, pulmonary PTLD is seen more often in lung transplant recipients than in patients who have undergone transplantation of other solid organs. In kidney transplant recipients, PTLD most commonly manifests in the allograft itself, gastrointestinal tract, central nervous system, or lymph nodes; lung involvement is less common. Chest imaging in affected patients may reveal nodular or reticulonodular infiltrates of basilar predominance, solitary or multiple masses, cavitating or necrotic lesions, and/or lymphadenopathy. In this patient who has undergone renal transplantation, late-onset PTLD with isolated pulmonary involvement, with only ground-glass opacities on lung imaging, would be an atypical presentation of an uncommon syndrome.

Despite empiric treatment with antibiotics and antiviral agents, the patient’s fever persisted. His respiratory rate increased to 30 breaths per minute. His hypoxia worsened, and he required nasal cannula high-flow oxygen supplementation at 30 L/min with a fraction of inspired oxygen (FiO₂) of 40%. On the fifth hospital day, contrast CT scan of the chest and abdomen showed new infiltrates in the bilateral upper lung fields as well as an area of low density in the tail of the pancreas without a focal mass (Figure 1). At this point, BAL was performed, and fluid PCR analysis returned positive for Pneumocystis jirovecii. CMV direct immunoperoxidase staining of leukocytes with peroxidase-labeled monoclonal antibody (C7-HRP test) was positive at five cells per 7.35 × 10⁴ peripheral blood leukocytes. The serum Epstein-Barr virus (EBV) viral capsid antigen (VCA) IgG was positive, while VCA IgM and EBV nuclear antigen IgG were negative. A bone marrow biopsy revealed mild hemophagocytosis. His serum soluble interleukin-2 (sIL2R) level was elevated at 5,254 U/mL (normal range, 122-496 U/mL). Given the BAL Pneumocystis PCR result, the dose of prednisolone was increased to 30 mg/day, and the patient’s fever subsided. Supplemental oxygen was weaned to an FiO₂ of 35%.

A CMV antigenemia assay returned positive, suggesting prior CMV infection. However, to diagnose CMV pneumonia, the virus must be detected in BAL fluid by PCR or cytologic analysis. CMV infection has been associated with cytopenias, HLH, pancreatic infiltration, and an increased risk for fungal infections and EBV-related PTLD. CMV infection could explain the first phase of this patient’s illness. Serum and BAL PCR for CMV are advised. Meanwhile, EBV testing indicates prior infection but does not distinguish between recent or more distant infection. EBV has been implicated in the pathophysiology of PTLD, as EBV-infected lymphoid tissue may proliferate in a variety of organs under reduced T-cell surveillance. EBV infection or PTLD with resulting immunomodulation may pose other explanations for this patient’s development of PCP infection. Cytologic analysis of the BAL fluid and marrow aspirate for evidence of PTLD is warranted. Finally, CMV, EBV, and PTLD have each been reported to trigger HLH. Though this patient has fevers, mild marrow hemophagocytosis, elevated serum ferritin, and elevated serum IL-2 receptor levels, he does not meet other diagnostic criteria for HLH (such as more pronounced cytopenias, splenomegaly, hypertriglyceridemia, hypofibrinogenemia, and low or absent natural killer T-cell activity). However, HLH may be muted in this patient because he was prescribed cyclosporine, which has been used in HLH treatment protocols.

On the 11^th hospital day, the patient developed hemorrhagic shock due to massive hematemesis and was transferred to the intensive care unit. His hemoglobin level was 5.9 g/dL. A total of 18 units of packed red blood cells were transfused over the following week for ongoing gastrointestinal bleeding. The serum LDH level increased to 4,139 IU/L, and the ferritin level rose to 7,855 ng/mL. The EBV copy level by serum PCR returned at 1 × 10⁶ copies/mL (normal range, less than 2 x 10² copies/mL). The patient was started on methylprednisolone (1 g/day for three days) and transitioned to dexamethasone and cyclosporine for possible EBV-related HLH. Ceftazidime, vancomycin, trimethoprim-sulfamethoxazole, and ciprofloxacin were administered. Amphotericin-B was initiated empirically for potential fungal pneumonia. Ganciclovir was continued. However, the patient remained in shock despite vasopressors and transfusions and died on the 22^nd hospital day.

The patient deteriorated despite broad antimicrobial therapy. Laboratory studies revealed EBV viremia and rising serum LDH. Recent EBV infection may have induced PTLD in the gastrointestinal tract, which is a commonly involved site among affected renal transplant patients. Corticosteroids and stress from critical illness can contribute to intestinal mucosal erosion and bleeding from a luminal PTLD lesion. Overall, the patient’s condition was likely explained by EBV infection, which triggered HLH and gastrointestinal PTLD. The resulting immunomodulation increased his risk for PCP infection beyond that conferred by chronic immunosuppression. It is still possible that he had concomitant CMV pneumonia, Aspergillus pneumonia, or even pulmonary PTLD, in addition to the proven PCP diagnosis.

An autopsy was performed. Atypical lymphocytic infiltration and diffuse alveolar damage were shown in the right upper lobe (Figure 2). EBV RNA-positive atypical lymphocytes coexpressing CD20 were demonstrated in multiple organs including the bone marrow, lungs, heart, stomach, adrenal glands, duodenum, ileum, and mesentery (Figure 3). This confirmed the diagnosis of an underlying EBV-positive posttransplant lymphoproliferative disorder. Serum and BAL CMV PCR assays returned negative. Neither CMV nor Aspergillus was identified in autopsy specimens.

A broad differential diagnosis should be considered when acute fever develops in a patient who has undergone solid organ transplantation. Causes may include opportunistic and nonopportunistic infections as well as noninfectious etiologies such as malignancy, organ rejection, inflammatory conditions, and medication toxicity.^1,2 As the discussant noted, more than one infection, or both infection and malignancy, can coexist in immunocompromised patients. For example, while viral pathogens such as EBV, CMV, and respiratory syncytial virus can cause illness due to direct tissue infection, they can also exert indirect effects in transplant recipients: acting as cofactors for and enabling other infections by causing immunosuppression (eg, Aspergillus or PCP developing after CMV infection), triggering graft rejection by upregulating proinflammatory cytokines, and inducing oncogenesis (eg, EBV-related PTLD).^1,3-5

PTLD is a rare but serious complication of solid organ transplantation and immunosuppression. Most cases are driven by EBV infection and subsequent transformation of infected lymphoid tissue in a variety of organs in the context of reduced T-cell surveillance.⁶ The incidence of PTLD varies based on the organ transplanted, ranging from 0.8%-2.5% in those who have undergone renal transplantation to 1.0%-5.5% in liver transplant recipients and 3.0%-10% in lung transplant recipients.³ The incidence has increased over the past decade. This may be due to a greater number of solid organ transplantations being performed, aging of the transplant donor/recipient population, new immunosuppressive regimens, and improved PTLD diagnosis due to superior diagnostic tools and clinician awareness.³ However, the mortality rate among solid organ transplant recipients with PTLD remains high, ranging from 40% to 70%.⁶

Risk factors for PTLD include a greater intensity of T-cell immunosuppression,⁷ history of pretransplant malignancy, recipient EBV seronegativity and donor seropositivity, and younger age at the time of transplantation.^8-10 EBV-related PTLD incidence in solid organ transplant recipients is greatest in the early posttransplantation course (the period of most intense immunosuppression) with over 80% of cases occurring in the first posttransplant year.¹¹

A high index of suspicion for PTLD is warranted in any solid organ transplant recipient who presents with constitutional symptoms, adenopathy, or cytopenias. Clinical suspicion of PTLD can be informed by risk factors, constitutional symptoms, elevated serum LDH, a detectable or rising serum EBV viral load, and radiologic adenopathy or visceral tissue infiltration.¹² The clinical presentation of PTLD is heterogeneous and varies in accordance with the organs affected. Extranodal involvement, such as pulmonary, gastrointestinal, and bone marrow involvement, is more common in PTLD than in other types of lymphoma.¹³ In this patient, the cytopenias, elevated serum LDH level, lung infiltrates, and radiologic pancreatic tail abnormality served as early clues to the presence of underlying PTLD.

The standard approach to diagnosing PTLD is biopsy of a suspicious lesion (adenopathy or an infiltrated visceral organ) with histopathological examination. Pathology may demonstrate distorted tissue architecture, clonal lymphocytes, or EBV-positive lymphocytes.¹⁴ Conventional CT is the most commonly used imaging modality to detect adenopathy or tissue infiltration related to PTLD,³ though 18F-fluorodeoxyglucose position-emission tomography (FDG-PET) is also used. Although FDG-PET has high diagnostic accuracy, with an overall sensitivity of 89% and specificity of 89%, false-negative results have been reported, particularly in cases of early PTLD lesions and diffuse large B-cell lymphoma.¹⁵ The majority of patients with EBV-associated PTLD demonstrate significant elevations in the serum EBV viral load compared with immunosuppressed controls without PTLD.¹⁶ An elevated EBV viral load can support a diagnosis of PTLD, though the absence of EBV viremia does not rule it out.¹⁷ Some transplant centers perform posttransplantation monitoring of the serum EBV viral load to aid in PTLD risk assessment and early diagnosis.

Management of PTLD is patient-specific and may involve reduction of immunosuppressive therapy, rituximab, chemotherapy, surgical excision, and/or radiation.¹³ Reduction of immunosuppression is the cornerstone of treatment.¹⁸ In patients who do not respond to the reduction of immunosuppression, rituximab and immunochemotherapy are second-line treatment options. A prospective, multicenter phase 2 trial (the PTLD-1 trial) demonstrated a complete response rate of 40% among patients with PTLD managed with rituximab.¹⁹

In summary, this case illustrates the importance of maintaining a broad differential diagnosis when acute fever develops in a patient who has undergone solid organ transplantation. The presence of more than one condition should be considered when the clinical presentation cannot be explained by a single diagnosis, as infections and malignancies can coexist in immunocompromised hosts. This case also highlights an unusual clinical presentation of PTLD, which was heralded mainly by its immunomodulatory effects rather than by compatible symptoms or obvious mass lesions.

Carefully reviewing the patient’s medical history and understanding how it sets the stage for the present illness is an essential step in clinical problem solving, because what is past is prologue.

• Fever in solid organ transplant recipients should prompt consideration of a broad differential diagnosis, including infection, malignancy, organ graft rejection, autoimmune disease, and medication toxicity.
• PTLD is a rare but serious complication of organ transplantation. Most cases are driven by EBV infection and transformation of infected lymphocytes in a variety of organs in the context of reduced T-cell surveillance. The clinical presentation can be heterogeneous and varies depending on the organs and tissues involved.
• More than one infection, or both infection and malignancy, can coexist in organ transplant recipients. Viral pathogens can exert direct pathologic effects on tissue but can also exert indirect effects, such as contributing to opportunistic infection susceptibility, graft rejection, and oncogenesis.

Disclosures

The authors have nothing to disclose.

Previous Publication

This case was originally reported in the 121st Okinawa Association of Medical Sciences in 2015 in Okinawa, Japan, and the conference abstracts were covered in The Okinawa Medical Journal. The publication did not provide any detailed, step-by-step analysis of clinical decision-making.

A 56-year-old Japanese man with a history of renal transplantation 20 years prior presented to the emergency department (ED) with two months of dyspnea on exertion and one day of fever and chills. The patient was in his usual state of health until two months prior to presentation, when he gradually noticed shortness of breath after sustained or effortful physical activities. The dyspnea improved with rest. Over the following two months, he noticed that the shortness of breath came on with lesser degrees of exertion, such as walking 100 meters. One day before presentation, he developed a fever of 39°C and chills at home, which prompted him to seek ED care. He denied chest pain, cough, leg swelling, or paroxysmal nocturnal dyspnea.

The differential diagnosis of exertional dyspnea progressing over several months includes cardiac, pulmonary, hematologic, and neuromuscular conditions. The patient’s history of renal transplantation prompts consideration of worsening indolent pneumonia (eg, Aspergillus, cytomegalovirus [CMV], or Pneumocystis pneumonia), allograft dysfunction with volume overload, recrudescence of the underlying disease that incited renal failure earlier in life (eg, vasculitis), or a late-onset posttransplantation lymphoproliferative disorder (PTLD). Additionally, acute fever in an immunocompromised patient immediately raises suspicion for infection (eg, pneumonia, enteritis, or urinary tract infection). At this point, it is difficult to know whether the subacute-to-chronic exertional dyspnea and the acute fever are consequences of the same disease or separate, potentially overlapping, processes.

His past medical history was significant for end-stage renal disease due to membranoproliferative glomerular nephropathy (MPGN), for which living, related-donor kidney transplantation was performed 20 years earlier. He also had type 2 diabetes mellitus, hypertension, and basal cell carcinoma of the face, which had been resected three years prior without spread or recurrence. He had no known allergies. Medications included prednisolone 15 mg daily, azathioprine 100 mg daily, and cyclosporine 100 mg daily, as well as amlodipine and candesartan. He lived in Japan with his wife and children. He denied any animal or environmental exposures. He did not smoke cigarettes or drink alcohol and had not traveled recently. His father had diabetes mellitus.

Recrudescence of an underlying autoimmune condition that may have incited MPGN earlier in life is unlikely while taking an immunosuppressive regimen consisting of prednisolone, azathioprine, and cyclosporine. However, these medications do increase susceptibility to infections, lymphoma, and skin cancers. Though he is immunocompromised, the patient is not on prophylaxis for Pneumocystis pneumonia (PCP). PCP in HIV-negative patients is associated with recent glucocorticoid exposure and typically follows an acute-to-subacute course with hypoxemia and respiratory distress. Though the risk of PCP infection is considered highest in the early posttransplantation period (when immunosuppression is most intense), many cases are diagnosed years after transplantation among patients no longer on prophylaxis. The patient has type 2 diabetes mellitus and hypertension, which are known complications of calcineurin inhibitor and steroid therapy and increase the risk of cardiovascular disease. Cardiovascular disease is a major cause of death among renal transplant recipients. Exertional dyspnea may be the presenting symptom of coronary artery disease.

On physical examination, the patient was alert, oriented, and in no acute distress. His temperature was 38.5°C, blood pressure 120/60 mm Hg, heart rate 146 beats per minute, respiratory rate 18 breaths per minute, and oxygen saturation 93% while breathing ambient air. The conjunctiva were normal without pallor or icterus. There was no cervical lymphadenopathy. Cardiac examination revealed tachycardia with a regular rhythm, normal S1 and S2, and no murmurs, rubs, or gallops. Jugular venous pressure was not elevated, and there was no lower extremity edema. Lungs were clear to auscultation bilaterally. The abdomen was soft, nontender, and nondistended. There was no tenderness over the transplanted kidney and no hepatosplenomegaly.

Dyspnea, fever, and tachycardia may be the sole manifestations of pneumonia in solid organ transplant recipients. The absence of cough or adventitious breath sounds does not eliminate concern for pneumonia. Pathogens that cause indolent pneumonia in immunocompromised patients include viruses (such as typical respiratory viruses and CMV), bacteria (typical organisms, Nocardia, Rhodococcus), and mycobacteria. Fungal causes include Aspergillus, Candida, Cryptococcus, Pneumocystis, and endemic mycoses. A detailed environmental history should be taken, and providers should ascertain which fungal diseases are endemic in the patient’s region of residence. There are no examination features suggesting hypervolemia or anemia. Although there is no hepatosplenomegaly or lymphadenopathy, PTLD often involves extranodal tissues, including the lungs. The incidence of PTLD is highest in the 12 months following transplantation, but it may occur at any time in the posttransplantation course. A complete blood count, comprehensive metabolic panel, lactate dehydrogenase (LDH), and blood and sputum cultures are indicated, along with computed tomography (CT) of the chest.

The leukocyte count was 3,500 cells/mm³, the hemoglobin level 9.0 g/dL, mean corpuscular volume 102 fL, and the platelet count 137,000 cells/mm³. The sodium level was 130 mEq/L, potassium 4.6 mEq/L, blood urea nitrogen 41 mg/dL, and creatinine 3.5 mg/dL. These complete blood count and serum electrolyte results were unchanged from the patient’s baseline values. The serum LDH level was 1,895 IU/L (normal range, 115-245 IU/L). The serum ferritin was 2,114 ng/mL (normal range, 13-277 ng/mL). A chest radiograph revealed diffuse, airspace-filling opacities in the bilateral lung bases. The urinalysis was normal. The patient was admitted and started empirically on intravenous ceftriaxone for potential bacterial pneumonia.

Chronic pancytopenia may result from azathioprine or cyclosporine use, marrow suppression or infiltration by a multisystem disease, or nutritional deficiency. Hemophagocytic lymphohistiocytosis (HLH) triggered by infection, a rheumatologic condition, acquired immunodeficiency, or malignancy can present with fevers, pancytopenia, and elevated ferritin, while splenomegaly may be absent. The euvolemic state, baseline creatinine level, and normal urinalysis argue against allograft dysfunction. The elevated serum ferritin nonspecifically confirms systemic inflammation. LDH, an intracellular enzyme involved in the bidirectional conversion of lactate to pyruvate, is expressed across tissue types. Elevated serum LDH attests to cell destruction, in this case potentially from lung infection (such as PCP) or malignancy (such as PTLD). At this point, the differential diagnosis of fever and pulmonary infiltrates in this patient remains broad.

Azathioprine and cyclosporine were stopped. The patient remained febrile despite the administration of intravenous antibiotics. His hypoxia worsened with an oxygen saturation of 90%-93% on 5 L/min of supplemental oxygen administered by nasal cannula. Noncontrast chest CT obtained on the second hospital day revealed ground-glass opacities in the bilateral lung bases. Blood, sputum, and urine cultures were sterile. As empiric therapies, ganciclovir was started for CMV infection, ciprofloxacin added for atypical pneumonia, and trimethoprim-sulfamethoxazole added for Pneumocystis infection.

These chest imaging findings help prioritize the differential diagnosis. Bibasilar ground-glass opacities are evident, while pulmonary masses, nodules, cavitation, adenopathy, and pleural effusions are absent. The differential diagnosis of multifocal ground-glass opacities on chest imaging includes infectious pneumonia, chronic interstitial lung disease, acute alveolar conditions (eg, cardiogenic pulmonary edema, acute respiratory distress syndrome, diffuse alveolar hemorrhage), or other pathologies (eg, drug toxicity, bronchoalveolar carcinoma, cryptogenic organizing pneumonia).

Infectious pneumonia is the principal concern. A diagnosis of PCP could be unifying, given dyspnea, progressive respiratory failure with hypoxia, and elevated LDH in an immunocompromised patient who is not prescribed PCP prophylaxis. The bilateral lung infiltrates and the absence of thoracic adenopathy or pleural effusions are characteristic of PCP as well. However, caution should be exercised in making specific infectious diagnoses in immunocompromised hosts on the basis of clinical and imaging findings alone. There can be overlap in the radiologic appearance of various infections (eg, CMV pneumonia can also present with bilateral ground-glass infiltrates, with concurrent fever, hypoxia, and pancytopenia). Additionally, more than one pneumonic pathogen may be implicated (eg, acute viral pneumonia superimposed on indolent fungal pneumonia). Polymerase chain reaction (PCR) analysis of respiratory secretions for viruses, serum PCR and serologic testing for herpes viruses, and serum beta-D-glucan and galactomannan assays are indicated. Serum serologic testing for fungi and bacteria such as Nocardia can be helpful, though the negative predictive values of these tests may be reduced in patients with impaired humoral immunity. Timely bronchoalveolar lavage (BAL) with microbiologic and PCR analysis and cytology is advised.

Fever, elevated LDH, cytopenias, and pulmonary infiltrates also raise suspicion for an underlying hematologic malignancy, such as PTLD. However, pulmonary PTLD is seen more often in lung transplant recipients than in patients who have undergone transplantation of other solid organs. In kidney transplant recipients, PTLD most commonly manifests in the allograft itself, gastrointestinal tract, central nervous system, or lymph nodes; lung involvement is less common. Chest imaging in affected patients may reveal nodular or reticulonodular infiltrates of basilar predominance, solitary or multiple masses, cavitating or necrotic lesions, and/or lymphadenopathy. In this patient who has undergone renal transplantation, late-onset PTLD with isolated pulmonary involvement, with only ground-glass opacities on lung imaging, would be an atypical presentation of an uncommon syndrome.

Despite empiric treatment with antibiotics and antiviral agents, the patient’s fever persisted. His respiratory rate increased to 30 breaths per minute. His hypoxia worsened, and he required nasal cannula high-flow oxygen supplementation at 30 L/min with a fraction of inspired oxygen (FiO₂) of 40%. On the fifth hospital day, contrast CT scan of the chest and abdomen showed new infiltrates in the bilateral upper lung fields as well as an area of low density in the tail of the pancreas without a focal mass (Figure 1). At this point, BAL was performed, and fluid PCR analysis returned positive for Pneumocystis jirovecii. CMV direct immunoperoxidase staining of leukocytes with peroxidase-labeled monoclonal antibody (C7-HRP test) was positive at five cells per 7.35 × 10⁴ peripheral blood leukocytes. The serum Epstein-Barr virus (EBV) viral capsid antigen (VCA) IgG was positive, while VCA IgM and EBV nuclear antigen IgG were negative. A bone marrow biopsy revealed mild hemophagocytosis. His serum soluble interleukin-2 (sIL2R) level was elevated at 5,254 U/mL (normal range, 122-496 U/mL). Given the BAL Pneumocystis PCR result, the dose of prednisolone was increased to 30 mg/day, and the patient’s fever subsided. Supplemental oxygen was weaned to an FiO₂ of 35%.

A CMV antigenemia assay returned positive, suggesting prior CMV infection. However, to diagnose CMV pneumonia, the virus must be detected in BAL fluid by PCR or cytologic analysis. CMV infection has been associated with cytopenias, HLH, pancreatic infiltration, and an increased risk for fungal infections and EBV-related PTLD. CMV infection could explain the first phase of this patient’s illness. Serum and BAL PCR for CMV are advised. Meanwhile, EBV testing indicates prior infection but does not distinguish between recent or more distant infection. EBV has been implicated in the pathophysiology of PTLD, as EBV-infected lymphoid tissue may proliferate in a variety of organs under reduced T-cell surveillance. EBV infection or PTLD with resulting immunomodulation may pose other explanations for this patient’s development of PCP infection. Cytologic analysis of the BAL fluid and marrow aspirate for evidence of PTLD is warranted. Finally, CMV, EBV, and PTLD have each been reported to trigger HLH. Though this patient has fevers, mild marrow hemophagocytosis, elevated serum ferritin, and elevated serum IL-2 receptor levels, he does not meet other diagnostic criteria for HLH (such as more pronounced cytopenias, splenomegaly, hypertriglyceridemia, hypofibrinogenemia, and low or absent natural killer T-cell activity). However, HLH may be muted in this patient because he was prescribed cyclosporine, which has been used in HLH treatment protocols.

On the 11^th hospital day, the patient developed hemorrhagic shock due to massive hematemesis and was transferred to the intensive care unit. His hemoglobin level was 5.9 g/dL. A total of 18 units of packed red blood cells were transfused over the following week for ongoing gastrointestinal bleeding. The serum LDH level increased to 4,139 IU/L, and the ferritin level rose to 7,855 ng/mL. The EBV copy level by serum PCR returned at 1 × 10⁶ copies/mL (normal range, less than 2 x 10² copies/mL). The patient was started on methylprednisolone (1 g/day for three days) and transitioned to dexamethasone and cyclosporine for possible EBV-related HLH. Ceftazidime, vancomycin, trimethoprim-sulfamethoxazole, and ciprofloxacin were administered. Amphotericin-B was initiated empirically for potential fungal pneumonia. Ganciclovir was continued. However, the patient remained in shock despite vasopressors and transfusions and died on the 22^nd hospital day.

The patient deteriorated despite broad antimicrobial therapy. Laboratory studies revealed EBV viremia and rising serum LDH. Recent EBV infection may have induced PTLD in the gastrointestinal tract, which is a commonly involved site among affected renal transplant patients. Corticosteroids and stress from critical illness can contribute to intestinal mucosal erosion and bleeding from a luminal PTLD lesion. Overall, the patient’s condition was likely explained by EBV infection, which triggered HLH and gastrointestinal PTLD. The resulting immunomodulation increased his risk for PCP infection beyond that conferred by chronic immunosuppression. It is still possible that he had concomitant CMV pneumonia, Aspergillus pneumonia, or even pulmonary PTLD, in addition to the proven PCP diagnosis.

An autopsy was performed. Atypical lymphocytic infiltration and diffuse alveolar damage were shown in the right upper lobe (Figure 2). EBV RNA-positive atypical lymphocytes coexpressing CD20 were demonstrated in multiple organs including the bone marrow, lungs, heart, stomach, adrenal glands, duodenum, ileum, and mesentery (Figure 3). This confirmed the diagnosis of an underlying EBV-positive posttransplant lymphoproliferative disorder. Serum and BAL CMV PCR assays returned negative. Neither CMV nor Aspergillus was identified in autopsy specimens.

A broad differential diagnosis should be considered when acute fever develops in a patient who has undergone solid organ transplantation. Causes may include opportunistic and nonopportunistic infections as well as noninfectious etiologies such as malignancy, organ rejection, inflammatory conditions, and medication toxicity.^1,2 As the discussant noted, more than one infection, or both infection and malignancy, can coexist in immunocompromised patients. For example, while viral pathogens such as EBV, CMV, and respiratory syncytial virus can cause illness due to direct tissue infection, they can also exert indirect effects in transplant recipients: acting as cofactors for and enabling other infections by causing immunosuppression (eg, Aspergillus or PCP developing after CMV infection), triggering graft rejection by upregulating proinflammatory cytokines, and inducing oncogenesis (eg, EBV-related PTLD).^1,3-5

PTLD is a rare but serious complication of solid organ transplantation and immunosuppression. Most cases are driven by EBV infection and subsequent transformation of infected lymphoid tissue in a variety of organs in the context of reduced T-cell surveillance.⁶ The incidence of PTLD varies based on the organ transplanted, ranging from 0.8%-2.5% in those who have undergone renal transplantation to 1.0%-5.5% in liver transplant recipients and 3.0%-10% in lung transplant recipients.³ The incidence has increased over the past decade. This may be due to a greater number of solid organ transplantations being performed, aging of the transplant donor/recipient population, new immunosuppressive regimens, and improved PTLD diagnosis due to superior diagnostic tools and clinician awareness.³ However, the mortality rate among solid organ transplant recipients with PTLD remains high, ranging from 40% to 70%.⁶

Risk factors for PTLD include a greater intensity of T-cell immunosuppression,⁷ history of pretransplant malignancy, recipient EBV seronegativity and donor seropositivity, and younger age at the time of transplantation.^8-10 EBV-related PTLD incidence in solid organ transplant recipients is greatest in the early posttransplantation course (the period of most intense immunosuppression) with over 80% of cases occurring in the first posttransplant year.¹¹

A high index of suspicion for PTLD is warranted in any solid organ transplant recipient who presents with constitutional symptoms, adenopathy, or cytopenias. Clinical suspicion of PTLD can be informed by risk factors, constitutional symptoms, elevated serum LDH, a detectable or rising serum EBV viral load, and radiologic adenopathy or visceral tissue infiltration.¹² The clinical presentation of PTLD is heterogeneous and varies in accordance with the organs affected. Extranodal involvement, such as pulmonary, gastrointestinal, and bone marrow involvement, is more common in PTLD than in other types of lymphoma.¹³ In this patient, the cytopenias, elevated serum LDH level, lung infiltrates, and radiologic pancreatic tail abnormality served as early clues to the presence of underlying PTLD.

The standard approach to diagnosing PTLD is biopsy of a suspicious lesion (adenopathy or an infiltrated visceral organ) with histopathological examination. Pathology may demonstrate distorted tissue architecture, clonal lymphocytes, or EBV-positive lymphocytes.¹⁴ Conventional CT is the most commonly used imaging modality to detect adenopathy or tissue infiltration related to PTLD,³ though 18F-fluorodeoxyglucose position-emission tomography (FDG-PET) is also used. Although FDG-PET has high diagnostic accuracy, with an overall sensitivity of 89% and specificity of 89%, false-negative results have been reported, particularly in cases of early PTLD lesions and diffuse large B-cell lymphoma.¹⁵ The majority of patients with EBV-associated PTLD demonstrate significant elevations in the serum EBV viral load compared with immunosuppressed controls without PTLD.¹⁶ An elevated EBV viral load can support a diagnosis of PTLD, though the absence of EBV viremia does not rule it out.¹⁷ Some transplant centers perform posttransplantation monitoring of the serum EBV viral load to aid in PTLD risk assessment and early diagnosis.

Management of PTLD is patient-specific and may involve reduction of immunosuppressive therapy, rituximab, chemotherapy, surgical excision, and/or radiation.¹³ Reduction of immunosuppression is the cornerstone of treatment.¹⁸ In patients who do not respond to the reduction of immunosuppression, rituximab and immunochemotherapy are second-line treatment options. A prospective, multicenter phase 2 trial (the PTLD-1 trial) demonstrated a complete response rate of 40% among patients with PTLD managed with rituximab.¹⁹

In summary, this case illustrates the importance of maintaining a broad differential diagnosis when acute fever develops in a patient who has undergone solid organ transplantation. The presence of more than one condition should be considered when the clinical presentation cannot be explained by a single diagnosis, as infections and malignancies can coexist in immunocompromised hosts. This case also highlights an unusual clinical presentation of PTLD, which was heralded mainly by its immunomodulatory effects rather than by compatible symptoms or obvious mass lesions.

Carefully reviewing the patient’s medical history and understanding how it sets the stage for the present illness is an essential step in clinical problem solving, because what is past is prologue.

• Fever in solid organ transplant recipients should prompt consideration of a broad differential diagnosis, including infection, malignancy, organ graft rejection, autoimmune disease, and medication toxicity.
• PTLD is a rare but serious complication of organ transplantation. Most cases are driven by EBV infection and transformation of infected lymphocytes in a variety of organs in the context of reduced T-cell surveillance. The clinical presentation can be heterogeneous and varies depending on the organs and tissues involved.
• More than one infection, or both infection and malignancy, can coexist in organ transplant recipients. Viral pathogens can exert direct pathologic effects on tissue but can also exert indirect effects, such as contributing to opportunistic infection susceptibility, graft rejection, and oncogenesis.

Disclosures

The authors have nothing to disclose.

Previous Publication

This case was originally reported in the 121st Okinawa Association of Medical Sciences in 2015 in Okinawa, Japan, and the conference abstracts were covered in The Okinawa Medical Journal. The publication did not provide any detailed, step-by-step analysis of clinical decision-making.

A 56-year-old Japanese man with a history of renal transplantation 20 years prior presented to the emergency department (ED) with two months of dyspnea on exertion and one day of fever and chills. The patient was in his usual state of health until two months prior to presentation, when he gradually noticed shortness of breath after sustained or effortful physical activities. The dyspnea improved with rest. Over the following two months, he noticed that the shortness of breath came on with lesser degrees of exertion, such as walking 100 meters. One day before presentation, he developed a fever of 39°C and chills at home, which prompted him to seek ED care. He denied chest pain, cough, leg swelling, or paroxysmal nocturnal dyspnea.

The differential diagnosis of exertional dyspnea progressing over several months includes cardiac, pulmonary, hematologic, and neuromuscular conditions. The patient’s history of renal transplantation prompts consideration of worsening indolent pneumonia (eg, Aspergillus, cytomegalovirus [CMV], or Pneumocystis pneumonia), allograft dysfunction with volume overload, recrudescence of the underlying disease that incited renal failure earlier in life (eg, vasculitis), or a late-onset posttransplantation lymphoproliferative disorder (PTLD). Additionally, acute fever in an immunocompromised patient immediately raises suspicion for infection (eg, pneumonia, enteritis, or urinary tract infection). At this point, it is difficult to know whether the subacute-to-chronic exertional dyspnea and the acute fever are consequences of the same disease or separate, potentially overlapping, processes.

His past medical history was significant for end-stage renal disease due to membranoproliferative glomerular nephropathy (MPGN), for which living, related-donor kidney transplantation was performed 20 years earlier. He also had type 2 diabetes mellitus, hypertension, and basal cell carcinoma of the face, which had been resected three years prior without spread or recurrence. He had no known allergies. Medications included prednisolone 15 mg daily, azathioprine 100 mg daily, and cyclosporine 100 mg daily, as well as amlodipine and candesartan. He lived in Japan with his wife and children. He denied any animal or environmental exposures. He did not smoke cigarettes or drink alcohol and had not traveled recently. His father had diabetes mellitus.

Recrudescence of an underlying autoimmune condition that may have incited MPGN earlier in life is unlikely while taking an immunosuppressive regimen consisting of prednisolone, azathioprine, and cyclosporine. However, these medications do increase susceptibility to infections, lymphoma, and skin cancers. Though he is immunocompromised, the patient is not on prophylaxis for Pneumocystis pneumonia (PCP). PCP in HIV-negative patients is associated with recent glucocorticoid exposure and typically follows an acute-to-subacute course with hypoxemia and respiratory distress. Though the risk of PCP infection is considered highest in the early posttransplantation period (when immunosuppression is most intense), many cases are diagnosed years after transplantation among patients no longer on prophylaxis. The patient has type 2 diabetes mellitus and hypertension, which are known complications of calcineurin inhibitor and steroid therapy and increase the risk of cardiovascular disease. Cardiovascular disease is a major cause of death among renal transplant recipients. Exertional dyspnea may be the presenting symptom of coronary artery disease.

On physical examination, the patient was alert, oriented, and in no acute distress. His temperature was 38.5°C, blood pressure 120/60 mm Hg, heart rate 146 beats per minute, respiratory rate 18 breaths per minute, and oxygen saturation 93% while breathing ambient air. The conjunctiva were normal without pallor or icterus. There was no cervical lymphadenopathy. Cardiac examination revealed tachycardia with a regular rhythm, normal S1 and S2, and no murmurs, rubs, or gallops. Jugular venous pressure was not elevated, and there was no lower extremity edema. Lungs were clear to auscultation bilaterally. The abdomen was soft, nontender, and nondistended. There was no tenderness over the transplanted kidney and no hepatosplenomegaly.

Dyspnea, fever, and tachycardia may be the sole manifestations of pneumonia in solid organ transplant recipients. The absence of cough or adventitious breath sounds does not eliminate concern for pneumonia. Pathogens that cause indolent pneumonia in immunocompromised patients include viruses (such as typical respiratory viruses and CMV), bacteria (typical organisms, Nocardia, Rhodococcus), and mycobacteria. Fungal causes include Aspergillus, Candida, Cryptococcus, Pneumocystis, and endemic mycoses. A detailed environmental history should be taken, and providers should ascertain which fungal diseases are endemic in the patient’s region of residence. There are no examination features suggesting hypervolemia or anemia. Although there is no hepatosplenomegaly or lymphadenopathy, PTLD often involves extranodal tissues, including the lungs. The incidence of PTLD is highest in the 12 months following transplantation, but it may occur at any time in the posttransplantation course. A complete blood count, comprehensive metabolic panel, lactate dehydrogenase (LDH), and blood and sputum cultures are indicated, along with computed tomography (CT) of the chest.

The leukocyte count was 3,500 cells/mm³, the hemoglobin level 9.0 g/dL, mean corpuscular volume 102 fL, and the platelet count 137,000 cells/mm³. The sodium level was 130 mEq/L, potassium 4.6 mEq/L, blood urea nitrogen 41 mg/dL, and creatinine 3.5 mg/dL. These complete blood count and serum electrolyte results were unchanged from the patient’s baseline values. The serum LDH level was 1,895 IU/L (normal range, 115-245 IU/L). The serum ferritin was 2,114 ng/mL (normal range, 13-277 ng/mL). A chest radiograph revealed diffuse, airspace-filling opacities in the bilateral lung bases. The urinalysis was normal. The patient was admitted and started empirically on intravenous ceftriaxone for potential bacterial pneumonia.

Chronic pancytopenia may result from azathioprine or cyclosporine use, marrow suppression or infiltration by a multisystem disease, or nutritional deficiency. Hemophagocytic lymphohistiocytosis (HLH) triggered by infection, a rheumatologic condition, acquired immunodeficiency, or malignancy can present with fevers, pancytopenia, and elevated ferritin, while splenomegaly may be absent. The euvolemic state, baseline creatinine level, and normal urinalysis argue against allograft dysfunction. The elevated serum ferritin nonspecifically confirms systemic inflammation. LDH, an intracellular enzyme involved in the bidirectional conversion of lactate to pyruvate, is expressed across tissue types. Elevated serum LDH attests to cell destruction, in this case potentially from lung infection (such as PCP) or malignancy (such as PTLD). At this point, the differential diagnosis of fever and pulmonary infiltrates in this patient remains broad.

Azathioprine and cyclosporine were stopped. The patient remained febrile despite the administration of intravenous antibiotics. His hypoxia worsened with an oxygen saturation of 90%-93% on 5 L/min of supplemental oxygen administered by nasal cannula. Noncontrast chest CT obtained on the second hospital day revealed ground-glass opacities in the bilateral lung bases. Blood, sputum, and urine cultures were sterile. As empiric therapies, ganciclovir was started for CMV infection, ciprofloxacin added for atypical pneumonia, and trimethoprim-sulfamethoxazole added for Pneumocystis infection.

These chest imaging findings help prioritize the differential diagnosis. Bibasilar ground-glass opacities are evident, while pulmonary masses, nodules, cavitation, adenopathy, and pleural effusions are absent. The differential diagnosis of multifocal ground-glass opacities on chest imaging includes infectious pneumonia, chronic interstitial lung disease, acute alveolar conditions (eg, cardiogenic pulmonary edema, acute respiratory distress syndrome, diffuse alveolar hemorrhage), or other pathologies (eg, drug toxicity, bronchoalveolar carcinoma, cryptogenic organizing pneumonia).

Infectious pneumonia is the principal concern. A diagnosis of PCP could be unifying, given dyspnea, progressive respiratory failure with hypoxia, and elevated LDH in an immunocompromised patient who is not prescribed PCP prophylaxis. The bilateral lung infiltrates and the absence of thoracic adenopathy or pleural effusions are characteristic of PCP as well. However, caution should be exercised in making specific infectious diagnoses in immunocompromised hosts on the basis of clinical and imaging findings alone. There can be overlap in the radiologic appearance of various infections (eg, CMV pneumonia can also present with bilateral ground-glass infiltrates, with concurrent fever, hypoxia, and pancytopenia). Additionally, more than one pneumonic pathogen may be implicated (eg, acute viral pneumonia superimposed on indolent fungal pneumonia). Polymerase chain reaction (PCR) analysis of respiratory secretions for viruses, serum PCR and serologic testing for herpes viruses, and serum beta-D-glucan and galactomannan assays are indicated. Serum serologic testing for fungi and bacteria such as Nocardia can be helpful, though the negative predictive values of these tests may be reduced in patients with impaired humoral immunity. Timely bronchoalveolar lavage (BAL) with microbiologic and PCR analysis and cytology is advised.

Fever, elevated LDH, cytopenias, and pulmonary infiltrates also raise suspicion for an underlying hematologic malignancy, such as PTLD. However, pulmonary PTLD is seen more often in lung transplant recipients than in patients who have undergone transplantation of other solid organs. In kidney transplant recipients, PTLD most commonly manifests in the allograft itself, gastrointestinal tract, central nervous system, or lymph nodes; lung involvement is less common. Chest imaging in affected patients may reveal nodular or reticulonodular infiltrates of basilar predominance, solitary or multiple masses, cavitating or necrotic lesions, and/or lymphadenopathy. In this patient who has undergone renal transplantation, late-onset PTLD with isolated pulmonary involvement, with only ground-glass opacities on lung imaging, would be an atypical presentation of an uncommon syndrome.

Despite empiric treatment with antibiotics and antiviral agents, the patient’s fever persisted. His respiratory rate increased to 30 breaths per minute. His hypoxia worsened, and he required nasal cannula high-flow oxygen supplementation at 30 L/min with a fraction of inspired oxygen (FiO₂) of 40%. On the fifth hospital day, contrast CT scan of the chest and abdomen showed new infiltrates in the bilateral upper lung fields as well as an area of low density in the tail of the pancreas without a focal mass (Figure 1). At this point, BAL was performed, and fluid PCR analysis returned positive for Pneumocystis jirovecii. CMV direct immunoperoxidase staining of leukocytes with peroxidase-labeled monoclonal antibody (C7-HRP test) was positive at five cells per 7.35 × 10⁴ peripheral blood leukocytes. The serum Epstein-Barr virus (EBV) viral capsid antigen (VCA) IgG was positive, while VCA IgM and EBV nuclear antigen IgG were negative. A bone marrow biopsy revealed mild hemophagocytosis. His serum soluble interleukin-2 (sIL2R) level was elevated at 5,254 U/mL (normal range, 122-496 U/mL). Given the BAL Pneumocystis PCR result, the dose of prednisolone was increased to 30 mg/day, and the patient’s fever subsided. Supplemental oxygen was weaned to an FiO₂ of 35%.

A CMV antigenemia assay returned positive, suggesting prior CMV infection. However, to diagnose CMV pneumonia, the virus must be detected in BAL fluid by PCR or cytologic analysis. CMV infection has been associated with cytopenias, HLH, pancreatic infiltration, and an increased risk for fungal infections and EBV-related PTLD. CMV infection could explain the first phase of this patient’s illness. Serum and BAL PCR for CMV are advised. Meanwhile, EBV testing indicates prior infection but does not distinguish between recent or more distant infection. EBV has been implicated in the pathophysiology of PTLD, as EBV-infected lymphoid tissue may proliferate in a variety of organs under reduced T-cell surveillance. EBV infection or PTLD with resulting immunomodulation may pose other explanations for this patient’s development of PCP infection. Cytologic analysis of the BAL fluid and marrow aspirate for evidence of PTLD is warranted. Finally, CMV, EBV, and PTLD have each been reported to trigger HLH. Though this patient has fevers, mild marrow hemophagocytosis, elevated serum ferritin, and elevated serum IL-2 receptor levels, he does not meet other diagnostic criteria for HLH (such as more pronounced cytopenias, splenomegaly, hypertriglyceridemia, hypofibrinogenemia, and low or absent natural killer T-cell activity). However, HLH may be muted in this patient because he was prescribed cyclosporine, which has been used in HLH treatment protocols.

On the 11^th hospital day, the patient developed hemorrhagic shock due to massive hematemesis and was transferred to the intensive care unit. His hemoglobin level was 5.9 g/dL. A total of 18 units of packed red blood cells were transfused over the following week for ongoing gastrointestinal bleeding. The serum LDH level increased to 4,139 IU/L, and the ferritin level rose to 7,855 ng/mL. The EBV copy level by serum PCR returned at 1 × 10⁶ copies/mL (normal range, less than 2 x 10² copies/mL). The patient was started on methylprednisolone (1 g/day for three days) and transitioned to dexamethasone and cyclosporine for possible EBV-related HLH. Ceftazidime, vancomycin, trimethoprim-sulfamethoxazole, and ciprofloxacin were administered. Amphotericin-B was initiated empirically for potential fungal pneumonia. Ganciclovir was continued. However, the patient remained in shock despite vasopressors and transfusions and died on the 22^nd hospital day.

The patient deteriorated despite broad antimicrobial therapy. Laboratory studies revealed EBV viremia and rising serum LDH. Recent EBV infection may have induced PTLD in the gastrointestinal tract, which is a commonly involved site among affected renal transplant patients. Corticosteroids and stress from critical illness can contribute to intestinal mucosal erosion and bleeding from a luminal PTLD lesion. Overall, the patient’s condition was likely explained by EBV infection, which triggered HLH and gastrointestinal PTLD. The resulting immunomodulation increased his risk for PCP infection beyond that conferred by chronic immunosuppression. It is still possible that he had concomitant CMV pneumonia, Aspergillus pneumonia, or even pulmonary PTLD, in addition to the proven PCP diagnosis.

An autopsy was performed. Atypical lymphocytic infiltration and diffuse alveolar damage were shown in the right upper lobe (Figure 2). EBV RNA-positive atypical lymphocytes coexpressing CD20 were demonstrated in multiple organs including the bone marrow, lungs, heart, stomach, adrenal glands, duodenum, ileum, and mesentery (Figure 3). This confirmed the diagnosis of an underlying EBV-positive posttransplant lymphoproliferative disorder. Serum and BAL CMV PCR assays returned negative. Neither CMV nor Aspergillus was identified in autopsy specimens.

A broad differential diagnosis should be considered when acute fever develops in a patient who has undergone solid organ transplantation. Causes may include opportunistic and nonopportunistic infections as well as noninfectious etiologies such as malignancy, organ rejection, inflammatory conditions, and medication toxicity.^1,2 As the discussant noted, more than one infection, or both infection and malignancy, can coexist in immunocompromised patients. For example, while viral pathogens such as EBV, CMV, and respiratory syncytial virus can cause illness due to direct tissue infection, they can also exert indirect effects in transplant recipients: acting as cofactors for and enabling other infections by causing immunosuppression (eg, Aspergillus or PCP developing after CMV infection), triggering graft rejection by upregulating proinflammatory cytokines, and inducing oncogenesis (eg, EBV-related PTLD).^1,3-5

PTLD is a rare but serious complication of solid organ transplantation and immunosuppression. Most cases are driven by EBV infection and subsequent transformation of infected lymphoid tissue in a variety of organs in the context of reduced T-cell surveillance.⁶ The incidence of PTLD varies based on the organ transplanted, ranging from 0.8%-2.5% in those who have undergone renal transplantation to 1.0%-5.5% in liver transplant recipients and 3.0%-10% in lung transplant recipients.³ The incidence has increased over the past decade. This may be due to a greater number of solid organ transplantations being performed, aging of the transplant donor/recipient population, new immunosuppressive regimens, and improved PTLD diagnosis due to superior diagnostic tools and clinician awareness.³ However, the mortality rate among solid organ transplant recipients with PTLD remains high, ranging from 40% to 70%.⁶

Risk factors for PTLD include a greater intensity of T-cell immunosuppression,⁷ history of pretransplant malignancy, recipient EBV seronegativity and donor seropositivity, and younger age at the time of transplantation.^8-10 EBV-related PTLD incidence in solid organ transplant recipients is greatest in the early posttransplantation course (the period of most intense immunosuppression) with over 80% of cases occurring in the first posttransplant year.¹¹

A high index of suspicion for PTLD is warranted in any solid organ transplant recipient who presents with constitutional symptoms, adenopathy, or cytopenias. Clinical suspicion of PTLD can be informed by risk factors, constitutional symptoms, elevated serum LDH, a detectable or rising serum EBV viral load, and radiologic adenopathy or visceral tissue infiltration.¹² The clinical presentation of PTLD is heterogeneous and varies in accordance with the organs affected. Extranodal involvement, such as pulmonary, gastrointestinal, and bone marrow involvement, is more common in PTLD than in other types of lymphoma.¹³ In this patient, the cytopenias, elevated serum LDH level, lung infiltrates, and radiologic pancreatic tail abnormality served as early clues to the presence of underlying PTLD.

The standard approach to diagnosing PTLD is biopsy of a suspicious lesion (adenopathy or an infiltrated visceral organ) with histopathological examination. Pathology may demonstrate distorted tissue architecture, clonal lymphocytes, or EBV-positive lymphocytes.¹⁴ Conventional CT is the most commonly used imaging modality to detect adenopathy or tissue infiltration related to PTLD,³ though 18F-fluorodeoxyglucose position-emission tomography (FDG-PET) is also used. Although FDG-PET has high diagnostic accuracy, with an overall sensitivity of 89% and specificity of 89%, false-negative results have been reported, particularly in cases of early PTLD lesions and diffuse large B-cell lymphoma.¹⁵ The majority of patients with EBV-associated PTLD demonstrate significant elevations in the serum EBV viral load compared with immunosuppressed controls without PTLD.¹⁶ An elevated EBV viral load can support a diagnosis of PTLD, though the absence of EBV viremia does not rule it out.¹⁷ Some transplant centers perform posttransplantation monitoring of the serum EBV viral load to aid in PTLD risk assessment and early diagnosis.

Management of PTLD is patient-specific and may involve reduction of immunosuppressive therapy, rituximab, chemotherapy, surgical excision, and/or radiation.¹³ Reduction of immunosuppression is the cornerstone of treatment.¹⁸ In patients who do not respond to the reduction of immunosuppression, rituximab and immunochemotherapy are second-line treatment options. A prospective, multicenter phase 2 trial (the PTLD-1 trial) demonstrated a complete response rate of 40% among patients with PTLD managed with rituximab.¹⁹

In summary, this case illustrates the importance of maintaining a broad differential diagnosis when acute fever develops in a patient who has undergone solid organ transplantation. The presence of more than one condition should be considered when the clinical presentation cannot be explained by a single diagnosis, as infections and malignancies can coexist in immunocompromised hosts. This case also highlights an unusual clinical presentation of PTLD, which was heralded mainly by its immunomodulatory effects rather than by compatible symptoms or obvious mass lesions.

Carefully reviewing the patient’s medical history and understanding how it sets the stage for the present illness is an essential step in clinical problem solving, because what is past is prologue.

• Fever in solid organ transplant recipients should prompt consideration of a broad differential diagnosis, including infection, malignancy, organ graft rejection, autoimmune disease, and medication toxicity.
• PTLD is a rare but serious complication of organ transplantation. Most cases are driven by EBV infection and transformation of infected lymphocytes in a variety of organs in the context of reduced T-cell surveillance. The clinical presentation can be heterogeneous and varies depending on the organs and tissues involved.
• More than one infection, or both infection and malignancy, can coexist in organ transplant recipients. Viral pathogens can exert direct pathologic effects on tissue but can also exert indirect effects, such as contributing to opportunistic infection susceptibility, graft rejection, and oncogenesis.

Disclosures

The authors have nothing to disclose.

Previous Publication

This case was originally reported in the 121st Okinawa Association of Medical Sciences in 2015 in Okinawa, Japan, and the conference abstracts were covered in The Okinawa Medical Journal. The publication did not provide any detailed, step-by-step analysis of clinical decision-making.

Approximately 400,000 lumbar punctures (LPs) are performed in the United States annually for either diagnostic workup or therapeutic relief.¹ Lumbar punctures are increasingly being performed in the United States, with an estimated 97,000 LPs performed on Medicare fee-for-service beneficiaries in 2011 alone, which is an increase of approximately 4,000 LPs in the same population from 1991.² Approximately 273,612 LPs were performed on hospitalized patients in the United States in 2010,¹ and the inpatient hospital setting is the most common site for LPs.^2,3

Many LPs are referred to radiologists who have access to imaging guidance to aid with needle insertion.² However, referrals to radiology delay performance of LPs, and delayed diagnosis of acute bacterial meningitis, the most common yet serious condition for which LPs are performed, is associated with increased morbidity and mortality.^4-8 Furthermore, although initiating empiric antibiotic treatment for suspected acute bacterial meningitis is recommended in some cases, doing so routinely can cause false-negative cerebrospinal fluid (CSF) culture results, complicating decisions about de-escalation and duration of antibiotics that could have been safely avoided by promptly performing an LP.⁹

Delaying the performance of LP has been associated with increased mortality.¹⁰ Demonstration of proficiency in performance of lumbar puncture is considered a core competency for hospitalists,¹¹ and with the increasing availability of point-of-care ultrasound, hospitalists can use ultrasound to guide performance of LPs at the bedside.¹² However, 30% of patients requiring LP in emergency departments have difficult-to-palpate lumbar spine landmarks,¹³ and lumbar puncture performed based on palpation of landmarks alone has been reported to fail or be traumatic in 28% of patients.¹⁴ Use of ultrasound guidance for lumbar puncture has been shown in randomized controlled trials to improve procedural success rates, while reducing the time to successful LP, needle passes, patient pain scores, and risk of a traumatic LP.^15-17

The purpose of this position statement is to review the literature and present consensus-based recommendations on the performance of ultrasound-guided LP in adult patients. This position statement does not mandate that hospitalists use ultrasound guidance for LP, nor does it establish ultrasound guidance as the standard of care for LP. Similar to previously published Society of Hospital Medicine (SHM) position statements,^12,18,19 this document presents recommendations with supporting evidence for the clinical outcomes, techniques, and training for using ultrasound guidance for LP. A manuscript describing the technique of ultrasound guidance for LPs has been previously published by some of the authors of this position statement.²⁰

Detailed methods are described in Appendix 1. The SHM Point-of-care Ultrasound (POCUS) Task Force was assembled to carry out this guideline development project under the direction of the SHM Board of Directors, Director of Education, and Education Committee. All expert panel members were physicians or advanced practice providers with expertise in POCUS. Expert panel members were divided into working group members, external peer reviewers, and a methodologist. All Task Force members were required to disclose any potential conflicts of interests (Appendix 2). The literature search was conducted in two independent phases. The first phase included literature searches conducted by the six working group members themselves. Key clinical questions and draft recommendations were then prepared. A systematic literature search was conducted by a medical librarian based on the findings of the initial literature search and draft recommendations. The Medline, Embase, CINAHL, and Cochrane medical databases were searched from 1975 to December 2015 initially. Google Scholar was also searched without limiters. Updated searches were conducted in November 2016, January 2018, and October 2018. The search strings are included in Appendix 3. All article abstracts were first screened for relevance by at least two members of the working group. Full-text versions of screened articles were reviewed, and articles on the use of ultrasound to guide LP were selected. In addition, the following article types were excluded: non-English language, nonhuman, age <18 years, meeting abstracts, meeting posters, narrative reviews, case reports, letters, and editorials. Moreover, studies focusing on the use of ultrasound guidance for spinal nerve root injections, regional anesthesia, and assessment of lumbar spine anatomy alone were excluded. All relevant systematic reviews, meta-analyses, randomized controlled trials, and observational studies of ultrasound-guided LP were screened and selected. Final article selection was based on working group consensus, and the selected literature was incorporated into the draft recommendations.

The Research and Development (RAND) Appropriateness Method that required panel judgment and consensus was used.²¹ The 27 voting members of the SHM POCUS Task Force reviewed and voted on the draft recommendations considering the following five transforming factors: (1) Problem priority and importance, (2) Level of quality of evidence, (3) Benefit/harm balance, (4) Benefit/burden balance, and (5) Certainty/concerns about PEAF (Preferences/Equity/Acceptability/Feasibility). Panel members participated in two rounds of electronic voting using an internet-based electronic data collection tool (REDCap™) in February 2018 and April 2018 (Appendix 4). Voting on appropriateness was conducted using a 9-point Likert scale. The three zones of the 9-point Likert scale were inappropriate (1-3 points), uncertain (4-6 points), and appropriate (7-9 points). The degree of consensus was assessed using the RAND algorithm (Appendix Figure 1 and Table 1). Establishing a recommendation required at least 70% agreement that a recommendation was “appropriate.” A strong recommendation required 80% of the votes within one integer of the median, following the RAND rules. Disagreement was defined as >30% of panelists voting outside of the zone of the median.

Recommendations were classified as strong or weak/conditional based on preset rules defining the panel’s level of consensus, which determined the wording of each recommendation (Table 2). The revised consensus-based recommendations underwent internal and external reviews by POCUS experts from different subspecialties. The final review of this position statement was performed by members of the SHM POCUS Task Force, SHM Education Committee, and SHM Executive Committee. The SHM Executive Committee endorsed this position statement in June 2018 before submission to the Journal of Hospital Medicine.

Literature Search

A total of 4,389 references were pooled from four different sources: a search by a certified medical librarian in December 2015 (3,212 citations) that was updated in November 2016 (380 citations), January 2018 (282 citations), and October 2018 (274 citations); working group members’ personal bibliographies and searches (31 citations); and a search focusing on ultrasound-guided LP training (210 citations). A total of 232 full-text articles were reviewed, and the final selection included 77 articles that were abstracted into a data table and incorporated into the draft recommendations. Details of the literature search strategy are presented in Appendix 3.

Four domains (clinical outcomes, technique, training, and knowledge gaps) with 16 draft recommendations were generated based on a review of the literature. Selected references were abstracted and assigned to each draft recommendation. Rationales for each recommendation were drafted citing supporting evidence. After two rounds of panel voting, five recommendations did not achieve agreement based on the RAND rules, one recommendation was combined with another recommendation during peer review, and 10 statements received final approval. The degree of consensus based on the median score and the dispersion of voting around the median are shown in Appendix 5. Nine statements were approved as strong recommendations, and one was approved as a conditional recommendation. Therefore, the final recommendation count was 10. The strength of the recommendation and degree of consensus for each recommendation are summarized in Table 1.

Terminology

LP is a procedure in which a spinal needle is introduced into the subarachnoid space for the purpose of collecting CSF for diagnostic evaluation and/or therapeutic relief.

Throughout this document, the phrases “ultrasound-guided” and “ultrasound guidance” refer to the use of ultrasound to mark a needle insertion site immediately before performing the procedure. This is also known as static ultrasound guidance. Real-time or dynamic ultrasound guidance refers to direct visualization of the needle tip as it traverses through the skin and soft tissues to reach the ligamentum flavum. Any reference to real-time ultrasound guidance is explicitly stated.

Clinical outcomes

1) When ultrasound equipment is available, along with providers who are appropriately trained to use it, we recommend that ultrasound guidance should be used for site selection of LPs to reduce the number of needle insertion attempts and needle redirections and increase the overall procedure success rates, especially in patients who are obese or have difficult-to-palpate landmarks.

Rationale. LPs have historically been performed by selecting a needle insertion site based on palpation of anatomical landmarks. However, an estimated 30% of patients requiring LP in emergency departments have lumbar spine landmarks that are difficult to palpate, most commonly due to obesity.¹³ Furthermore, lumbar puncture performed based on palpation of landmarks alone has been reported to fail in 28% of patients.¹⁴

Ultrasound can be used at the bedside to elucidate the lumbar spine anatomy to guide performance of LP or epidural catheterization. Since the early 2000s, randomized studies comparing the use of ultrasound guidance (ultrasound-guided) versus anatomical landmarks (landmark-guided) to map the lumbar spine for epidural catheterization have emerged. It is important to recognize that the exact same ultrasound technique is used for site marking of LP, epidural catheterization, and spinal anesthesia—the key difference is how deep the needle tip is inserted. Therefore, data from these three ultrasound-guided procedures are often pooled. Currently, at least 33 randomized controlled studies comparing ultrasound-guided vs landmark-guided site selection for LP, epidural catheterization, or spinal anesthesia have been published.^22-49 We present three meta-analyses below that pooled data primarily from randomized controlled studies comparing ultrasound-guided vs landmark-guided site selection for LP or spinal anesthesia.

In 2013, Shaikh et al. published the first meta-analysis with 14 randomized controlled studies comparing ultrasound-guided vs landmark-guided site selection for LP (n = 5) or epidural catheterization (n = 9). The pooled data showed that use of ultrasound guidance decreased the proportion of failed procedures (risk ratio 0.21, 95% CI 0.10-0.43) with an absolute risk reduction of 6.3% (95% CI 4.1%-8.4%) and a number needed to treat of 16 (95% CI 12-25) to prevent one failed procedure. In addition, the use of ultrasound reduced the mean number of attempts by 0.44 (95% CI 0.24-0.64) and reduced the mean number of needle redirections by 1.00 (95% CI 0.75-1.24). The reduction in risk of a failed procedure was similar for LPs (risk ratio 0.19 [95% CI 0.07-0.56]) and epidural catheterizations (risk ratio 0.23 [95% CI 0.09-0.60]).¹⁶

A similar meta-analysis published by Perlas et al. in 2016 included a total of 31 studies, both randomized controlled and cohort studies, evaluating the use of ultrasound guidance for LP, spinal anesthesia, and epidural catheterization.⁵⁰ The goal of this systematic review and meta-analysis was to establish clinical practice recommendations. The authors concluded (1) the data consistently suggest that ultrasound is more accurate than palpation for lumbar interspace identification, (2) ultrasound allows accurate measurement of the needle insertion depth to reach the epidural space with a mean difference of <3 mm compared with the actual needle insertion depth, and (3) ultrasound increases the efficacy of lumbar epidural or spinal anesthesia by decreasing the mean number of needle passes for success by 0.75 (95% CI 0.44-1.07) and reducing the risk of a failed procedure (risk ratio 0.51 [95% CI 0.32-0.80]), both in patients with normal surface anatomy and in those with technically difficult surface anatomy due to obesity, scoliosis, or previous spine surgery.

Compared to the two earlier meta-analyses that included studies of both LP and spinal anesthesia procedures, the meta-analysis conducted by Gottlieb et al. in 2018 pooled data from 12 randomized controlled studies of ultrasound guidance for LPs only. For the primary outcome, pooled data from both adult and pediatric studies demonstrated higher procedural success rates with ultrasound-guided vs landmark-guided LPs (90% vs 81%) with an odds ratio of 2.1 (95% CI 0.66-7.44) in favor of ultrasound; however, there were no statistically significant differences when the adult and pediatric subgroups were analyzed separately, probably due to underpowering. For the secondary outcomes, data from the adult subgroup showed that use of ultrasound guidance was associated with fewer traumatic LPs (OR 0.28, 95% CI 0.14-0.59), shorter time to procedural success (adjusted mean difference –3.03 minutes, 95% CI –3.54 to –2.52), fewer number of needle passes (adjusted mean difference –0.81 passes, 95% CI –1.57 to –0.05), and lower patient pain scores (adjusted mean difference –2.53, 95% CI –3.89 to –1.17).

At least 12 randomized controlled studies have been published comparing the use of ultrasound guidance vs landmarks for the performance of LP or spinal anesthesia in adult patients, which were not included in the abovementioned meta-analyses. These individual studies demonstrated similar benefits of using ultrasound guidance: reduced needle insertion attempts, reduced needle redirections, and increased overall procedural success rates.^{17,31,37,40,41,43-49}

It is important to recognize that four randomized controlled studies did not demonstrate any benefits of ultrasound guidance on the number of attempts or procedural success rates,^23,33,41,51 and three of these studies were included in the abovementioned meta-analyses.^23,33,51 Limitations of these negative studies include potential selection bias, inadequate sample sizes, and varying levels of operator skills in procedures, ultrasound guidance, or both. One study included emergency medicine residents as operators with varying degrees of ultrasound skills, and more importantly, patient enrollment occurred by convenience sampling, which may have introduced selection bias. Furthermore, most of the patients were not obese (median BMI of 27 kg/m²), and it is unclear why 10 years lapsed from data collection until publication.³³ Another study with three experienced anesthesiologists as operators performing spinal anesthesia enrolled only patients who were not obese (mean BMI of 29 kg/m²) and had easily palpable bony landmarks—two patient characteristics associated with the least benefit of using ultrasound guidance in other studies.²³ Another negative study had one experienced anesthesiologist marking obstetric patients with ultrasound, but junior residents performing the actual procedure in the absence of the anesthesiologist who had marked the patient.⁴¹

In general, the greatest benefit of using ultrasound guidance for LP has been demonstrated in obese patients.^{24,32,34,35,52,53} Benefits have been shown in specific obese patient populations, including obstetric,^31,54,55 orthopedic,^24,56,57 and emergency department patients.³⁰

By increasing the procedural success rates with the use of ultrasound at the bedside, fewer patients may be referred to interventional radiology for fluoroscopic-guided LP, decreasing the patient exposure to ionizing radiation. A randomized study (n = 112) that compared site marking with ultrasound guidance versus fluoroscopic guidance for epidural steroid injections found the two techniques to be equivalent with respect to mean procedure time, number of needle insertion attempts, or needle passes.⁵⁸ Another randomized study found that the performance time of ultrasound guidance was two minutes shorter (P < .05) than fluoroscopic guidance.⁵⁹

Techniques

2) We recommend that ultrasound should be used to more accurately identify the lumbar spine level than physical examination in both obese and nonobese patients.

Rationale. Traditionally, an imaginary line connecting the iliac crests (intercristal line, Tuffier’s line, or Jacoby’s line) was considered to identify the L4 vertebra or the L4-L5 interspinous space in the midline; however, studies have revealed this traditional landmark to be much less accurate than previously thought. In general, palpating the iliac crests to mark the intercristal line identifies an interspinous space that is one space cephalad (ie, the L2-L3 interspinous space) but can range from L1-L2 to L4-L5.^46,60-64 If an LP is inadvertently performed in the L1-L2 interspinous space, the risk of spinal cord injury is higher than that when performed in a more distal interspinous space.

A study by Margarido et al. with 45 patients with a mean BMI of 30 kg/m² found that the intercristal line was located above the L4-L5 interspinous space in 100% of patients. More importantly, the intercristal line was above L2-L3 in 36% of patients and above L1-L2 in 4% of patients. It is important to note that patients with scoliosis or previous spine surgery were excluded from this study, and all examinations were performed by two experienced anesthesiologists with patients in a sitting position—all factors that would favor accurate palpation and marking of the iliac crests.⁶⁰

In a study of nonobese patients (mean BMI 28 kg/m²) undergoing spinal anesthesia, Duniec et al. compared the lumbar level identified by palpation versus ultrasound and found discordance between the two techniques in 36% of patients; 18% were one space too cephalad, 16% were one space too caudal, and 2% were off by two interspinous spaces.⁶¹ Another study found discordance in 64% of patients (mean BMI 28 kg/m²) when comparing the interspinous level where spinal anesthesia had been performed by palpation versus a post-procedural ultrasound examination. This study revealed that the interspinous space was more cephalad in 50% of patients with 6% of punctures performed in the L1-L2 interspace.⁶² A similar study compared the accuracy of palpation vs ultrasound to identify the L3-L4 interspinous space in obese (mean BMI 34 kg/m²) versus nonobese (mean BMI 27 kg/m²) patients. This study found marking a space above L3-L4 in 51% of obese and 40% of nonobese patients and marking of the L1-L2 interspace in 7% of obese and 4% of nonobese patients.⁶⁴

A study comparing palpation vs ultrasound found that 68% of obese patients with a BMI of >30 kg/m² had difficult-to-palpate lumbar spine landmarks, but with the use of ultrasound, landmarks were identified in 76% of all patients, including obese and nonobese, with difficult-to-palpate landmarks.⁶⁵

3) We suggest using ultrasound for selecting and marking a needle insertion site just before performing LPs in either a lateral decubitus or sitting position. The patient should remain in the same position after marking the needle insertion site.

Rationale. Ultrasound mapping of the lumbar spine can be performed in either a lateral decubitus or sitting position. Selecting and marking a needle insertion site should be performed at the bedside just before performing the procedure. The patient must remain in the same position in the interim between marking and inserting the needle, as a slight change in position can alter the needle trajectory, lowering the LP success rate. Although performing LPs in a lateral decubitus position has the advantage of accurately measuring the opening pressure, misalignment of the shoulder and pelvic girdles and bowing of the bed in a lateral decubitus position may lower LP success rates.

One randomized study comparing ultrasound-guided spinal anesthesia in a lateral decubitus versus sitting position found no difference in the number of needle insertion attempts or measurement of the skin-dura distance; however, the needle insertion depth was 0.73 cm greater in a lateral decubitus vs sitting position (P = .002).⁶⁶ Procedural success rates of LP with ultrasound guidance have not been directly compared in a sitting versus lateral decubitus position, although the overall procedural success rates were higher in one study that allowed the operator to choose either sitting or lateral decubitus position when ultrasound was used.³²

4) We recommend that a low-frequency transducer, preferably a curvilinear array transducer, should be used to evaluate the lumbar spine and mark a needle insertion site in most patients. A high-frequency linear array transducer may be used in nonobese patients.

Rationale. Low-frequency transducers emit sound waves that penetrate deep tissues, allowing visualization of bones and ligaments of the lumbar spine. A high-frequency linear transducer offers better resolution but shallower penetration to approximately 6-9 cm, limiting its use for site marking in overweight and obese patients. In obese patients, the ligamentum flavum is often deeper than 6 cm, which requires a low-frequency transducer to be visualized.

Most of the randomized controlled studies demonstrating benefits of using ultrasound guidance compared with landmark guidance for performance of LP, epidural anesthesia, or spinal anesthesia have used a low-frequency, curvilinear transducer.^{22,24,26-28,31,34-36,39,43-45,67} Two randomized controlled trials used a high-frequency linear transducer for site marking of lumbar procedures.^30,32,37 Using a high-frequency linear transducer has been described in real-time, ultrasound-guided LPs, the advantage being better needle visualization with a linear transducer.²⁹ Detection of blood vessels by color flow Doppler may be another advantage of using a high-frequency linear transducer, although a study by Grau et al. showed that use of color flow Doppler with a low-frequency curvilinear transducer permitted visualization of interspinous vessels as small as 0.5 mm in size.⁶⁸

5) We recommend that ultrasound should be used to map the lumbar spine, starting at the level of the sacrum and sliding the transducer cephalad, sequentially identifying the lumbar spine interspaces.Rationale. Although no studies have directly compared different ultrasound scanning protocols to map the lumbar spine, starting at the level of the sacrum and sliding the transducer cephalad to sequentially identify the lumbar interspinous spaces is the most commonly described technique in studies demonstrating improved clinical outcomes with the use of ultrasound.^{24,31,34,37,39,40,45,56,57,67} Because the sacrum can be easily recognized, identifying it first is most beneficial in patients with few or no palpable landmarks.

All five lumbar spinous processes and interspinous spaces can be mapped from the sacrum using either a midline or a paramedian approach, and the widest interspinous space can be selected. In a midline approach, either a transverse or a longitudinal view is obtained. The transducer is centered on the sacrum and slid cephalad from L5 to L1 to identify each spinous process and interspinous space. In a paramedian approach, longitudinal paramedian views are obtained from the L5–sacrum interspace to the L1–L2 interspace, and each interspinous space is identified as the transducer is slid cephalad. Both these approaches are effective for mapping the lumbar spine. Whether the entire lumbar spine is mapped, and whether a midline or a paramedian approach is utilized, will depend on the operator’s preference.

6) We recommend that ultrasound should be used in a transverse plane to mark the midline of the lumbar spine and a longitudinal plane to mark the interspinous spaces. The intersection of these two lines marks the needle insertion site.

Rationale. The most common technique described in comparative studies of ultrasound vs landmarks includes visualization of the lumbar spine in two planes, a transverse plane to identify the midline and a longitudinal plane to identify the interspinous spaces. The majority of randomized controlled studies that demonstrated a reduction in the number of needle insertion attempts and an increase in the procedural success rates have used this technique (see Clinical Outcomes).^{22,24,28,32,35-37,43,44} Marking the midline and interspinous space(s) for LP may be performed in any order, starting with either the transverse or longitudinal plane first.

The midline of the spine is marked by placing the transducer in a transverse plane over the lumbar spine, centering over the spinous processes that have a distinct hyperechoic tip and a prominent acoustic shadow deep to the bone, and drawing a line perpendicular to the center of the transducer delineating the midline. The midline should be marked over a minimum of two or three spinous processes.

To identify the interspinous spaces, the transducer is aligned longitudinally over the midline. The transducer is slid along the midline to identify the widest interspinous space. Once the transducer is centered over the widest interspinous space, a line perpendicular to the center of the transducer is drawn to mark the interspinous space. The intersection of the lines marking the spinal midline and the selected interspinous space identifies the needle entry point.

To visualize the ligamentum flavum from a paramedian view, the transducer is oriented longitudinally over the midline, slid approximately 1 cm laterally, and tilted approximately 15 degrees aiming the ultrasound beam toward the midline. The skin–ligamentum flavum distance is most reliably measured from a paramedian view. Alternatively, in some patients, the ligamentum flavum may be visualized in the midline and the depth can be measured.

7) We recommend that ultrasound should be used during a preprocedural evaluation to measure the distance from the skin surface to the ligamentum flavum from a longitudinal paramedian view to estimate the needle insertion depth and ensure that a spinal needle of adequate length is used.

Rationale. The distance from the skin to the ligamentum flavum can be measured using ultrasound during preprocedural planning. Knowing the depth to the ligamentum flavum preprocedurally allows the operator to procure a spinal needle of adequate length, anticipate the insertion depth before CSF can be obtained, determine the depth to which a local anesthetic will need to be injected, and decide whether the anticipated difficulty of the procedure warrants referral to or consultation with another specialist.

The skin–ligamentum flavum distance can be measured from a transverse midline view or a longitudinal paramedian view. A longitudinal paramedian view provides an unobstructed view of the ligamentum flavum due to less shadowing from bony structures compared with a midline view. Several studies have demonstrated a strong correlation between the skin–ligamentum flavum distance measured by ultrasound and the actual needle insertion depth in both midline and paramedian views.^{28,34,36,53,54,57,69,70}

A meta-analysis that included 13 comparative studies evaluating the correlation between ultrasound-measured depth and actual needle insertion depth to reach the epidural or intrathecal space consistently demonstrated a strong correlation between the measured and actual depth.⁵⁰ A few studies have reported near-perfect Pearson correlation coefficients of 0.98.^55,71,72 The pooled correlation was 0.91 (95% CI 0.87-0.94). All studies measured the depth from the skin to the ventral side of the ligamentum flavum or the intrathecal space from either a longitudinal paramedian view (n = 4) or a transverse midline view (n = 9). Eight of the more recent studies evaluated the accuracy of the ultrasound measurements and found the depth measurements by ultrasound to be accurate within 1-13 mm of the actual needle insertion depth, with seven of the eight studies reporting a mean difference of ≤3 mm.⁵⁰

Measurement of the distance between the skin and the ligamentum flavum generally underestimates the needle insertion depth. One study reported that measurement of the skin–ligamentum flavum distance underestimates the needle insertion depth by 7.6 mm to obtain CSF, whereas measurement of the skin–posterior longitudinal ligament distance overestimates the needle insertion depth by 2.5 mm.⁵⁷ A well-accepted contributor to underestimation of the depth measurements using ultrasound is compression of the skin and soft tissues by the transducer, and therefore, pressure on the skin must be released before freezing an image and measuring the depth to the subarachnoid space.

Training

8) We recommend that novices should undergo simulation-based training, where available, before attempting ultrasound-guided LPs on actual patients.

Rationale. Similar to training for other bedside procedures, dedicated training sessions, including didactics, supervised practice on patients, and simulation-based practice, should be considered when teaching novices to perform ultrasound-guided LP. Simulation-based training facilitates acquisition of knowledge and skills to perform invasive bedside procedures, including LP.⁷³ Simulation-based training has been commonly incorporated into procedure training for trainees using an immersive experience, such as a “boot camp,”^74-77 or a standardized curriculum,^78,79 and has demonstrated improvements in post-course procedural knowledge, technical skills, and operator confidence. Two of these studies included training in the use of ultrasound guidance for LP. These studies showed that simulation-based practice improved skill acquisition and confidence.^80,81 Simulation using novel computer software may improve skill acquisition in the use of ultrasound guidance for LP.⁸²

9) We recommend that training in ultrasound-guided LPs should be adapted based on prior ultrasound experience, as learning curves will vary.Rationale. The learning curve to achieve competency in the use of ultrasound guidance for LP has not been well studied. The rate of attaining competency in identifying lumbar spine structures using ultrasound will vary by provider based on prior skills in ultrasound-guided procedures.⁸³ Thus, providers with prior ultrasound experience may require less training than those without such experience to achieve competency. However, extensive experience in performing landmark-guided LPs does not necessarily translate into rapid acquisition of skills to perform the procedure with ultrasound guidance. A study of practicing anesthesiologists with no prior ultrasound experience demonstrated that 20 supervised trials of ultrasound-guided spinal anesthesia were insufficient to achieve competency.⁸⁴ Although minimums may be a necessary step to gain competence, using them as a sole means to define competence does not account for variable learning curves.¹² Based on a national survey of 21 hospitalist procedure experts, the mean current vs suggested minimums for initial and ongoing hospital privileging for LPs were 1.8 vs 6.9 and 2.2 vs 4.6 annually in one report.⁸⁵

A fundamental question that needs to be answered is how to define competency in the use of ultrasound guidance for LP, including the specific skills and knowledge that must be mastered. At a minimum, providers must be able to identify lumbar spinous processes and distinguish them from the sacrum, identify the lumbar interspinous spaces and their corresponding levels, and estimate the depth from the skin to the ligamentum flavum from the midline and paramedian planes. Novice operators may benefit from practicing lumbar spine mapping of nonobese patients using a high-frequency linear transducer that generates high-resolution images and facilitates recognition of lumbar spine structures.

10) We recommend that novice providers should be supervised when performing ultrasound-guided LPs before performing the procedure independently on patients.

Rationale: Demonstration of competency in the use of ultrasound to identify lumbar spine anatomy should be achieved before routinely performing the procedure independently on patients.¹⁸ All providers will require a variable period of supervised practice to demonstrate the appropriate technique, followed by a period of unsupervised practice before competency is achieved. Supervised practice with guidance and feedback has been shown to significantly improve providers’ ability to delineate lumbar spine anatomy.⁸⁶

The process of producing these guidelines revealed areas of uncertainty and important gaps in the literature regarding the use of ultrasound guidance for LP.

First, it is unclear whether the use of ultrasound guidance for LP reduces postprocedural back pain and whether it improves patient satisfaction. Several studies have evaluated postprocedural back pain^{28,30,32,33,52} and patient satisfaction^28,29,33,51 with the use of ultrasound guidance, but these studies have found inconsistent results. Some of these results were probably due to insufficient statistical power or confounding variables. Furthermore, benefits have been demonstrated in certain subgroups, such as overweight patients or those with anatomical abnormalities, as was found in two studies.^52,87 Use of ultrasound guidance for spinal anesthesia has been shown to reduce postprocedural headache²⁸ and improve patient satisfaction⁵¹, although similar benefit has not been demonstrated in patients undergoing LP.

Second, the effect of using ultrasound guidance on the frequency of traumatic LPs is an area of uncertainty. A “traumatic tap” is defined as an inadvertent puncture of an epidural vein during passage of the spinal needle through the dura. It remains difficult to discern in these studies whether red blood cells detected in the CSF resulted from puncture of an epidural vein or from needle trauma of the skin and soft tissues. Despite this uncertainty, at least seven randomized controlled studies have assessed the effect of ultrasound guidance on traumatic LPs. The meta-analysis by Shaikh et al. included five randomized controlled studies that assessed the effect of ultrasound guidance on the reporting of traumatic taps. The study found a reduced risk of traumatic taps (risk ratio 0.27 [95% CI 0.11-0.67]), an absolute risk reduction of 5.9% (95% CI 2.3%-9.5%), and a number needed to treat of 17 (95% CI 11-44) to prevent one traumatic tap.¹⁶ Similarly, the meta-analysis by Gottlieb et al. showed a lower risk of traumatic taps among adults undergoing LP with ultrasound guidance in five randomized controlled studies with an odds ratio of 0.28 (95% CI 0.14-0.59). The meta-analysis by Gottlieb et al. included two adult studies that were not included by Shaikh et al.

Third, several important questions about the technique of ultrasound-guided LP remain unanswered. In addition to the static technique, a dynamic technique with real-time needle tracking has been described to perform ultrasound-guided LP, epidural catheterization, and spinal anesthesia. A pilot study by Grau et al. found that ultrasound used either statically or dynamically had fewer insertion attempts and needle redirections than use of landmarks alone.²⁹ Three other pilot studies showed successful spinal anesthesia in almost all patients^88-90 and one large study demonstrated successful spinal anesthesia with real-time ultrasound guidance in 97 of 100 patients with a median of three needle passes.⁹¹ Furthermore, a few industry-sponsored studies with small numbers of patients have described the use of novel needle tracking systems that facilitate needle visualization during real-time ultrasound-guided LP.^92,93 However, to our knowledge, no comparative studies of static versus dynamic guidance using novel needle tracking systems in human subjects have been published, and any potential role for these novel needle tracking systems has not yet been defined.

Finally, the effects of using ultrasound guidance on clinical decision-making, timeliness, and cost-effectiveness of LP have not yet been explored but could have important clinical practice implications.

Randomized controlled trials have demonstrated that using ultrasound guidance for LPs can reduce the number of needle insertion attempts and needle redirections and increase the overall procedural success rates. Ultrasound can more accurately identify the lumbar spine level than physical examination in both obese and nonobese patients, although the greatest benefit of using ultrasound guidance for LPs has been shown in obese patients.

Ultrasound permits assessment of the interspinous space width and measurement of the ligamentum flavum depth to select an optimal needle insertion site and adequate length spinal needle. Although the use of real-time ultrasound guidance has been described, the use of static ultrasound guidance for LP site marking remains the standard technique.

Acknowledgments

The authors thank all the members of the Society of Hospital Medicine Point-of-care Ultrasound Task Force and the Education Committee members for their time and dedication to develop these guidelines.

Collaborators from Society of Hospital Medicine Point-of-care Ultrasound Task Force: Saaid Abdel-Ghani, Robert Arntfield, Jeffrey Bates, Anjali Bhagra, Michael Blaivas, Daniel Brotman, Carolina Candotti, Richard Hoppmann, Susan Hunt, Trevor P. Jensen, Paul Mayo, Benji Mathews, Satyen Nichani, Vicki Noble, Martin Perez, Nitin Puri, Aliaksei Pustavoitau, Kreegan Reierson, Sophia Rodgers, Kirk Spencer, Vivek Tayal, David Tierney

SHM Point-of-care Ultrasound Task Force: CHAIRS: Nilam Soni, Ricardo Franco-Sadud, Jeff Bates. WORKING GROUPS: Thoracentesis Working Group: Ria Dancel (chair), Daniel Schnobrich, Nitin Puri. Vascular Access Working Group: Ricardo Franco (chair), Benji Matthews, Saaid Abdel-Ghani, Sophia Rodgers, Martin Perez, Daniel Schnobrich. Paracentesis Working Group: Joel Cho (chair), Benji Matthews, Kreegan Reierson, Anjali Bhagra, Trevor P. Jensen Lumbar Puncture Working Group: Nilam J. Soni (chair), Ricardo Franco, Gerard Salame, Josh Lenchus, Venkat Kalidindi, Ketino Kobaidze. Credentialing Working Group: Brian P Lucas (chair), David Tierney, Trevor P. Jensen PEER REVIEWERS: Robert Arntfield, Michael Blaivas, Richard Hoppmann, Paul Mayo, Vicki Noble, Aliaksei Pustavoitau, Kirk Spencer, Vivek Tayal. METHODOLOGIST: Mahmoud El Barbary. LIBRARIAN: Loretta Grikis. SOCIETY OF HOSPITAL MEDICINE EDUCATION COMMITTEE: Daniel Brotman (past chair), Satyen Nichani (current chair), Susan Hunt. SOCIETY OF HOSPITAL MEDICINE STAFF: Nick Marzano.

Disclosures

The authors have nothing to disclose.

Funding

Brian P Lucas: Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development and Dartmouth SYNERGY, National Institutes of Health, National Center for Translational Science (UL1TR001086). Nilam Soni: Department of Veterans Affairs, Quality Enhancement Research Initiative (QUERI) Partnered Evaluation Initiative Grant (HX002263-01A1).

Disclaimer

The contents of this publication do not represent the views of the U.S. Department of Veterans Affairs or the United States Government.

Approximately 400,000 lumbar punctures (LPs) are performed in the United States annually for either diagnostic workup or therapeutic relief.¹ Lumbar punctures are increasingly being performed in the United States, with an estimated 97,000 LPs performed on Medicare fee-for-service beneficiaries in 2011 alone, which is an increase of approximately 4,000 LPs in the same population from 1991.² Approximately 273,612 LPs were performed on hospitalized patients in the United States in 2010,¹ and the inpatient hospital setting is the most common site for LPs.^2,3

Many LPs are referred to radiologists who have access to imaging guidance to aid with needle insertion.² However, referrals to radiology delay performance of LPs, and delayed diagnosis of acute bacterial meningitis, the most common yet serious condition for which LPs are performed, is associated with increased morbidity and mortality.^4-8 Furthermore, although initiating empiric antibiotic treatment for suspected acute bacterial meningitis is recommended in some cases, doing so routinely can cause false-negative cerebrospinal fluid (CSF) culture results, complicating decisions about de-escalation and duration of antibiotics that could have been safely avoided by promptly performing an LP.⁹

Delaying the performance of LP has been associated with increased mortality.¹⁰ Demonstration of proficiency in performance of lumbar puncture is considered a core competency for hospitalists,¹¹ and with the increasing availability of point-of-care ultrasound, hospitalists can use ultrasound to guide performance of LPs at the bedside.¹² However, 30% of patients requiring LP in emergency departments have difficult-to-palpate lumbar spine landmarks,¹³ and lumbar puncture performed based on palpation of landmarks alone has been reported to fail or be traumatic in 28% of patients.¹⁴ Use of ultrasound guidance for lumbar puncture has been shown in randomized controlled trials to improve procedural success rates, while reducing the time to successful LP, needle passes, patient pain scores, and risk of a traumatic LP.^15-17

The purpose of this position statement is to review the literature and present consensus-based recommendations on the performance of ultrasound-guided LP in adult patients. This position statement does not mandate that hospitalists use ultrasound guidance for LP, nor does it establish ultrasound guidance as the standard of care for LP. Similar to previously published Society of Hospital Medicine (SHM) position statements,^12,18,19 this document presents recommendations with supporting evidence for the clinical outcomes, techniques, and training for using ultrasound guidance for LP. A manuscript describing the technique of ultrasound guidance for LPs has been previously published by some of the authors of this position statement.²⁰

Detailed methods are described in Appendix 1. The SHM Point-of-care Ultrasound (POCUS) Task Force was assembled to carry out this guideline development project under the direction of the SHM Board of Directors, Director of Education, and Education Committee. All expert panel members were physicians or advanced practice providers with expertise in POCUS. Expert panel members were divided into working group members, external peer reviewers, and a methodologist. All Task Force members were required to disclose any potential conflicts of interests (Appendix 2). The literature search was conducted in two independent phases. The first phase included literature searches conducted by the six working group members themselves. Key clinical questions and draft recommendations were then prepared. A systematic literature search was conducted by a medical librarian based on the findings of the initial literature search and draft recommendations. The Medline, Embase, CINAHL, and Cochrane medical databases were searched from 1975 to December 2015 initially. Google Scholar was also searched without limiters. Updated searches were conducted in November 2016, January 2018, and October 2018. The search strings are included in Appendix 3. All article abstracts were first screened for relevance by at least two members of the working group. Full-text versions of screened articles were reviewed, and articles on the use of ultrasound to guide LP were selected. In addition, the following article types were excluded: non-English language, nonhuman, age <18 years, meeting abstracts, meeting posters, narrative reviews, case reports, letters, and editorials. Moreover, studies focusing on the use of ultrasound guidance for spinal nerve root injections, regional anesthesia, and assessment of lumbar spine anatomy alone were excluded. All relevant systematic reviews, meta-analyses, randomized controlled trials, and observational studies of ultrasound-guided LP were screened and selected. Final article selection was based on working group consensus, and the selected literature was incorporated into the draft recommendations.

The Research and Development (RAND) Appropriateness Method that required panel judgment and consensus was used.²¹ The 27 voting members of the SHM POCUS Task Force reviewed and voted on the draft recommendations considering the following five transforming factors: (1) Problem priority and importance, (2) Level of quality of evidence, (3) Benefit/harm balance, (4) Benefit/burden balance, and (5) Certainty/concerns about PEAF (Preferences/Equity/Acceptability/Feasibility). Panel members participated in two rounds of electronic voting using an internet-based electronic data collection tool (REDCap™) in February 2018 and April 2018 (Appendix 4). Voting on appropriateness was conducted using a 9-point Likert scale. The three zones of the 9-point Likert scale were inappropriate (1-3 points), uncertain (4-6 points), and appropriate (7-9 points). The degree of consensus was assessed using the RAND algorithm (Appendix Figure 1 and Table 1). Establishing a recommendation required at least 70% agreement that a recommendation was “appropriate.” A strong recommendation required 80% of the votes within one integer of the median, following the RAND rules. Disagreement was defined as >30% of panelists voting outside of the zone of the median.

Recommendations were classified as strong or weak/conditional based on preset rules defining the panel’s level of consensus, which determined the wording of each recommendation (Table 2). The revised consensus-based recommendations underwent internal and external reviews by POCUS experts from different subspecialties. The final review of this position statement was performed by members of the SHM POCUS Task Force, SHM Education Committee, and SHM Executive Committee. The SHM Executive Committee endorsed this position statement in June 2018 before submission to the Journal of Hospital Medicine.

Literature Search

A total of 4,389 references were pooled from four different sources: a search by a certified medical librarian in December 2015 (3,212 citations) that was updated in November 2016 (380 citations), January 2018 (282 citations), and October 2018 (274 citations); working group members’ personal bibliographies and searches (31 citations); and a search focusing on ultrasound-guided LP training (210 citations). A total of 232 full-text articles were reviewed, and the final selection included 77 articles that were abstracted into a data table and incorporated into the draft recommendations. Details of the literature search strategy are presented in Appendix 3.

Four domains (clinical outcomes, technique, training, and knowledge gaps) with 16 draft recommendations were generated based on a review of the literature. Selected references were abstracted and assigned to each draft recommendation. Rationales for each recommendation were drafted citing supporting evidence. After two rounds of panel voting, five recommendations did not achieve agreement based on the RAND rules, one recommendation was combined with another recommendation during peer review, and 10 statements received final approval. The degree of consensus based on the median score and the dispersion of voting around the median are shown in Appendix 5. Nine statements were approved as strong recommendations, and one was approved as a conditional recommendation. Therefore, the final recommendation count was 10. The strength of the recommendation and degree of consensus for each recommendation are summarized in Table 1.

Terminology

LP is a procedure in which a spinal needle is introduced into the subarachnoid space for the purpose of collecting CSF for diagnostic evaluation and/or therapeutic relief.

Throughout this document, the phrases “ultrasound-guided” and “ultrasound guidance” refer to the use of ultrasound to mark a needle insertion site immediately before performing the procedure. This is also known as static ultrasound guidance. Real-time or dynamic ultrasound guidance refers to direct visualization of the needle tip as it traverses through the skin and soft tissues to reach the ligamentum flavum. Any reference to real-time ultrasound guidance is explicitly stated.

Clinical outcomes

1) When ultrasound equipment is available, along with providers who are appropriately trained to use it, we recommend that ultrasound guidance should be used for site selection of LPs to reduce the number of needle insertion attempts and needle redirections and increase the overall procedure success rates, especially in patients who are obese or have difficult-to-palpate landmarks.

Rationale. LPs have historically been performed by selecting a needle insertion site based on palpation of anatomical landmarks. However, an estimated 30% of patients requiring LP in emergency departments have lumbar spine landmarks that are difficult to palpate, most commonly due to obesity.¹³ Furthermore, lumbar puncture performed based on palpation of landmarks alone has been reported to fail in 28% of patients.¹⁴

Ultrasound can be used at the bedside to elucidate the lumbar spine anatomy to guide performance of LP or epidural catheterization. Since the early 2000s, randomized studies comparing the use of ultrasound guidance (ultrasound-guided) versus anatomical landmarks (landmark-guided) to map the lumbar spine for epidural catheterization have emerged. It is important to recognize that the exact same ultrasound technique is used for site marking of LP, epidural catheterization, and spinal anesthesia—the key difference is how deep the needle tip is inserted. Therefore, data from these three ultrasound-guided procedures are often pooled. Currently, at least 33 randomized controlled studies comparing ultrasound-guided vs landmark-guided site selection for LP, epidural catheterization, or spinal anesthesia have been published.^22-49 We present three meta-analyses below that pooled data primarily from randomized controlled studies comparing ultrasound-guided vs landmark-guided site selection for LP or spinal anesthesia.

In 2013, Shaikh et al. published the first meta-analysis with 14 randomized controlled studies comparing ultrasound-guided vs landmark-guided site selection for LP (n = 5) or epidural catheterization (n = 9). The pooled data showed that use of ultrasound guidance decreased the proportion of failed procedures (risk ratio 0.21, 95% CI 0.10-0.43) with an absolute risk reduction of 6.3% (95% CI 4.1%-8.4%) and a number needed to treat of 16 (95% CI 12-25) to prevent one failed procedure. In addition, the use of ultrasound reduced the mean number of attempts by 0.44 (95% CI 0.24-0.64) and reduced the mean number of needle redirections by 1.00 (95% CI 0.75-1.24). The reduction in risk of a failed procedure was similar for LPs (risk ratio 0.19 [95% CI 0.07-0.56]) and epidural catheterizations (risk ratio 0.23 [95% CI 0.09-0.60]).¹⁶

A similar meta-analysis published by Perlas et al. in 2016 included a total of 31 studies, both randomized controlled and cohort studies, evaluating the use of ultrasound guidance for LP, spinal anesthesia, and epidural catheterization.⁵⁰ The goal of this systematic review and meta-analysis was to establish clinical practice recommendations. The authors concluded (1) the data consistently suggest that ultrasound is more accurate than palpation for lumbar interspace identification, (2) ultrasound allows accurate measurement of the needle insertion depth to reach the epidural space with a mean difference of <3 mm compared with the actual needle insertion depth, and (3) ultrasound increases the efficacy of lumbar epidural or spinal anesthesia by decreasing the mean number of needle passes for success by 0.75 (95% CI 0.44-1.07) and reducing the risk of a failed procedure (risk ratio 0.51 [95% CI 0.32-0.80]), both in patients with normal surface anatomy and in those with technically difficult surface anatomy due to obesity, scoliosis, or previous spine surgery.

Compared to the two earlier meta-analyses that included studies of both LP and spinal anesthesia procedures, the meta-analysis conducted by Gottlieb et al. in 2018 pooled data from 12 randomized controlled studies of ultrasound guidance for LPs only. For the primary outcome, pooled data from both adult and pediatric studies demonstrated higher procedural success rates with ultrasound-guided vs landmark-guided LPs (90% vs 81%) with an odds ratio of 2.1 (95% CI 0.66-7.44) in favor of ultrasound; however, there were no statistically significant differences when the adult and pediatric subgroups were analyzed separately, probably due to underpowering. For the secondary outcomes, data from the adult subgroup showed that use of ultrasound guidance was associated with fewer traumatic LPs (OR 0.28, 95% CI 0.14-0.59), shorter time to procedural success (adjusted mean difference –3.03 minutes, 95% CI –3.54 to –2.52), fewer number of needle passes (adjusted mean difference –0.81 passes, 95% CI –1.57 to –0.05), and lower patient pain scores (adjusted mean difference –2.53, 95% CI –3.89 to –1.17).

At least 12 randomized controlled studies have been published comparing the use of ultrasound guidance vs landmarks for the performance of LP or spinal anesthesia in adult patients, which were not included in the abovementioned meta-analyses. These individual studies demonstrated similar benefits of using ultrasound guidance: reduced needle insertion attempts, reduced needle redirections, and increased overall procedural success rates.^{17,31,37,40,41,43-49}

It is important to recognize that four randomized controlled studies did not demonstrate any benefits of ultrasound guidance on the number of attempts or procedural success rates,^23,33,41,51 and three of these studies were included in the abovementioned meta-analyses.^23,33,51 Limitations of these negative studies include potential selection bias, inadequate sample sizes, and varying levels of operator skills in procedures, ultrasound guidance, or both. One study included emergency medicine residents as operators with varying degrees of ultrasound skills, and more importantly, patient enrollment occurred by convenience sampling, which may have introduced selection bias. Furthermore, most of the patients were not obese (median BMI of 27 kg/m²), and it is unclear why 10 years lapsed from data collection until publication.³³ Another study with three experienced anesthesiologists as operators performing spinal anesthesia enrolled only patients who were not obese (mean BMI of 29 kg/m²) and had easily palpable bony landmarks—two patient characteristics associated with the least benefit of using ultrasound guidance in other studies.²³ Another negative study had one experienced anesthesiologist marking obstetric patients with ultrasound, but junior residents performing the actual procedure in the absence of the anesthesiologist who had marked the patient.⁴¹

In general, the greatest benefit of using ultrasound guidance for LP has been demonstrated in obese patients.^{24,32,34,35,52,53} Benefits have been shown in specific obese patient populations, including obstetric,^31,54,55 orthopedic,^24,56,57 and emergency department patients.³⁰

By increasing the procedural success rates with the use of ultrasound at the bedside, fewer patients may be referred to interventional radiology for fluoroscopic-guided LP, decreasing the patient exposure to ionizing radiation. A randomized study (n = 112) that compared site marking with ultrasound guidance versus fluoroscopic guidance for epidural steroid injections found the two techniques to be equivalent with respect to mean procedure time, number of needle insertion attempts, or needle passes.⁵⁸ Another randomized study found that the performance time of ultrasound guidance was two minutes shorter (P < .05) than fluoroscopic guidance.⁵⁹

Techniques

2) We recommend that ultrasound should be used to more accurately identify the lumbar spine level than physical examination in both obese and nonobese patients.

Rationale. Traditionally, an imaginary line connecting the iliac crests (intercristal line, Tuffier’s line, or Jacoby’s line) was considered to identify the L4 vertebra or the L4-L5 interspinous space in the midline; however, studies have revealed this traditional landmark to be much less accurate than previously thought. In general, palpating the iliac crests to mark the intercristal line identifies an interspinous space that is one space cephalad (ie, the L2-L3 interspinous space) but can range from L1-L2 to L4-L5.^46,60-64 If an LP is inadvertently performed in the L1-L2 interspinous space, the risk of spinal cord injury is higher than that when performed in a more distal interspinous space.

A study by Margarido et al. with 45 patients with a mean BMI of 30 kg/m² found that the intercristal line was located above the L4-L5 interspinous space in 100% of patients. More importantly, the intercristal line was above L2-L3 in 36% of patients and above L1-L2 in 4% of patients. It is important to note that patients with scoliosis or previous spine surgery were excluded from this study, and all examinations were performed by two experienced anesthesiologists with patients in a sitting position—all factors that would favor accurate palpation and marking of the iliac crests.⁶⁰

In a study of nonobese patients (mean BMI 28 kg/m²) undergoing spinal anesthesia, Duniec et al. compared the lumbar level identified by palpation versus ultrasound and found discordance between the two techniques in 36% of patients; 18% were one space too cephalad, 16% were one space too caudal, and 2% were off by two interspinous spaces.⁶¹ Another study found discordance in 64% of patients (mean BMI 28 kg/m²) when comparing the interspinous level where spinal anesthesia had been performed by palpation versus a post-procedural ultrasound examination. This study revealed that the interspinous space was more cephalad in 50% of patients with 6% of punctures performed in the L1-L2 interspace.⁶² A similar study compared the accuracy of palpation vs ultrasound to identify the L3-L4 interspinous space in obese (mean BMI 34 kg/m²) versus nonobese (mean BMI 27 kg/m²) patients. This study found marking a space above L3-L4 in 51% of obese and 40% of nonobese patients and marking of the L1-L2 interspace in 7% of obese and 4% of nonobese patients.⁶⁴

A study comparing palpation vs ultrasound found that 68% of obese patients with a BMI of >30 kg/m² had difficult-to-palpate lumbar spine landmarks, but with the use of ultrasound, landmarks were identified in 76% of all patients, including obese and nonobese, with difficult-to-palpate landmarks.⁶⁵

3) We suggest using ultrasound for selecting and marking a needle insertion site just before performing LPs in either a lateral decubitus or sitting position. The patient should remain in the same position after marking the needle insertion site.

Rationale. Ultrasound mapping of the lumbar spine can be performed in either a lateral decubitus or sitting position. Selecting and marking a needle insertion site should be performed at the bedside just before performing the procedure. The patient must remain in the same position in the interim between marking and inserting the needle, as a slight change in position can alter the needle trajectory, lowering the LP success rate. Although performing LPs in a lateral decubitus position has the advantage of accurately measuring the opening pressure, misalignment of the shoulder and pelvic girdles and bowing of the bed in a lateral decubitus position may lower LP success rates.

One randomized study comparing ultrasound-guided spinal anesthesia in a lateral decubitus versus sitting position found no difference in the number of needle insertion attempts or measurement of the skin-dura distance; however, the needle insertion depth was 0.73 cm greater in a lateral decubitus vs sitting position (P = .002).⁶⁶ Procedural success rates of LP with ultrasound guidance have not been directly compared in a sitting versus lateral decubitus position, although the overall procedural success rates were higher in one study that allowed the operator to choose either sitting or lateral decubitus position when ultrasound was used.³²

4) We recommend that a low-frequency transducer, preferably a curvilinear array transducer, should be used to evaluate the lumbar spine and mark a needle insertion site in most patients. A high-frequency linear array transducer may be used in nonobese patients.

Rationale. Low-frequency transducers emit sound waves that penetrate deep tissues, allowing visualization of bones and ligaments of the lumbar spine. A high-frequency linear transducer offers better resolution but shallower penetration to approximately 6-9 cm, limiting its use for site marking in overweight and obese patients. In obese patients, the ligamentum flavum is often deeper than 6 cm, which requires a low-frequency transducer to be visualized.

Most of the randomized controlled studies demonstrating benefits of using ultrasound guidance compared with landmark guidance for performance of LP, epidural anesthesia, or spinal anesthesia have used a low-frequency, curvilinear transducer.^{22,24,26-28,31,34-36,39,43-45,67} Two randomized controlled trials used a high-frequency linear transducer for site marking of lumbar procedures.^30,32,37 Using a high-frequency linear transducer has been described in real-time, ultrasound-guided LPs, the advantage being better needle visualization with a linear transducer.²⁹ Detection of blood vessels by color flow Doppler may be another advantage of using a high-frequency linear transducer, although a study by Grau et al. showed that use of color flow Doppler with a low-frequency curvilinear transducer permitted visualization of interspinous vessels as small as 0.5 mm in size.⁶⁸

5) We recommend that ultrasound should be used to map the lumbar spine, starting at the level of the sacrum and sliding the transducer cephalad, sequentially identifying the lumbar spine interspaces.Rationale. Although no studies have directly compared different ultrasound scanning protocols to map the lumbar spine, starting at the level of the sacrum and sliding the transducer cephalad to sequentially identify the lumbar interspinous spaces is the most commonly described technique in studies demonstrating improved clinical outcomes with the use of ultrasound.^{24,31,34,37,39,40,45,56,57,67} Because the sacrum can be easily recognized, identifying it first is most beneficial in patients with few or no palpable landmarks.

All five lumbar spinous processes and interspinous spaces can be mapped from the sacrum using either a midline or a paramedian approach, and the widest interspinous space can be selected. In a midline approach, either a transverse or a longitudinal view is obtained. The transducer is centered on the sacrum and slid cephalad from L5 to L1 to identify each spinous process and interspinous space. In a paramedian approach, longitudinal paramedian views are obtained from the L5–sacrum interspace to the L1–L2 interspace, and each interspinous space is identified as the transducer is slid cephalad. Both these approaches are effective for mapping the lumbar spine. Whether the entire lumbar spine is mapped, and whether a midline or a paramedian approach is utilized, will depend on the operator’s preference.

6) We recommend that ultrasound should be used in a transverse plane to mark the midline of the lumbar spine and a longitudinal plane to mark the interspinous spaces. The intersection of these two lines marks the needle insertion site.

Rationale. The most common technique described in comparative studies of ultrasound vs landmarks includes visualization of the lumbar spine in two planes, a transverse plane to identify the midline and a longitudinal plane to identify the interspinous spaces. The majority of randomized controlled studies that demonstrated a reduction in the number of needle insertion attempts and an increase in the procedural success rates have used this technique (see Clinical Outcomes).^{22,24,28,32,35-37,43,44} Marking the midline and interspinous space(s) for LP may be performed in any order, starting with either the transverse or longitudinal plane first.

The midline of the spine is marked by placing the transducer in a transverse plane over the lumbar spine, centering over the spinous processes that have a distinct hyperechoic tip and a prominent acoustic shadow deep to the bone, and drawing a line perpendicular to the center of the transducer delineating the midline. The midline should be marked over a minimum of two or three spinous processes.

To identify the interspinous spaces, the transducer is aligned longitudinally over the midline. The transducer is slid along the midline to identify the widest interspinous space. Once the transducer is centered over the widest interspinous space, a line perpendicular to the center of the transducer is drawn to mark the interspinous space. The intersection of the lines marking the spinal midline and the selected interspinous space identifies the needle entry point.

To visualize the ligamentum flavum from a paramedian view, the transducer is oriented longitudinally over the midline, slid approximately 1 cm laterally, and tilted approximately 15 degrees aiming the ultrasound beam toward the midline. The skin–ligamentum flavum distance is most reliably measured from a paramedian view. Alternatively, in some patients, the ligamentum flavum may be visualized in the midline and the depth can be measured.

7) We recommend that ultrasound should be used during a preprocedural evaluation to measure the distance from the skin surface to the ligamentum flavum from a longitudinal paramedian view to estimate the needle insertion depth and ensure that a spinal needle of adequate length is used.

Rationale. The distance from the skin to the ligamentum flavum can be measured using ultrasound during preprocedural planning. Knowing the depth to the ligamentum flavum preprocedurally allows the operator to procure a spinal needle of adequate length, anticipate the insertion depth before CSF can be obtained, determine the depth to which a local anesthetic will need to be injected, and decide whether the anticipated difficulty of the procedure warrants referral to or consultation with another specialist.

The skin–ligamentum flavum distance can be measured from a transverse midline view or a longitudinal paramedian view. A longitudinal paramedian view provides an unobstructed view of the ligamentum flavum due to less shadowing from bony structures compared with a midline view. Several studies have demonstrated a strong correlation between the skin–ligamentum flavum distance measured by ultrasound and the actual needle insertion depth in both midline and paramedian views.^{28,34,36,53,54,57,69,70}

A meta-analysis that included 13 comparative studies evaluating the correlation between ultrasound-measured depth and actual needle insertion depth to reach the epidural or intrathecal space consistently demonstrated a strong correlation between the measured and actual depth.⁵⁰ A few studies have reported near-perfect Pearson correlation coefficients of 0.98.^55,71,72 The pooled correlation was 0.91 (95% CI 0.87-0.94). All studies measured the depth from the skin to the ventral side of the ligamentum flavum or the intrathecal space from either a longitudinal paramedian view (n = 4) or a transverse midline view (n = 9). Eight of the more recent studies evaluated the accuracy of the ultrasound measurements and found the depth measurements by ultrasound to be accurate within 1-13 mm of the actual needle insertion depth, with seven of the eight studies reporting a mean difference of ≤3 mm.⁵⁰

Measurement of the distance between the skin and the ligamentum flavum generally underestimates the needle insertion depth. One study reported that measurement of the skin–ligamentum flavum distance underestimates the needle insertion depth by 7.6 mm to obtain CSF, whereas measurement of the skin–posterior longitudinal ligament distance overestimates the needle insertion depth by 2.5 mm.⁵⁷ A well-accepted contributor to underestimation of the depth measurements using ultrasound is compression of the skin and soft tissues by the transducer, and therefore, pressure on the skin must be released before freezing an image and measuring the depth to the subarachnoid space.

Training

8) We recommend that novices should undergo simulation-based training, where available, before attempting ultrasound-guided LPs on actual patients.

Rationale. Similar to training for other bedside procedures, dedicated training sessions, including didactics, supervised practice on patients, and simulation-based practice, should be considered when teaching novices to perform ultrasound-guided LP. Simulation-based training facilitates acquisition of knowledge and skills to perform invasive bedside procedures, including LP.⁷³ Simulation-based training has been commonly incorporated into procedure training for trainees using an immersive experience, such as a “boot camp,”^74-77 or a standardized curriculum,^78,79 and has demonstrated improvements in post-course procedural knowledge, technical skills, and operator confidence. Two of these studies included training in the use of ultrasound guidance for LP. These studies showed that simulation-based practice improved skill acquisition and confidence.^80,81 Simulation using novel computer software may improve skill acquisition in the use of ultrasound guidance for LP.⁸²

9) We recommend that training in ultrasound-guided LPs should be adapted based on prior ultrasound experience, as learning curves will vary.Rationale. The learning curve to achieve competency in the use of ultrasound guidance for LP has not been well studied. The rate of attaining competency in identifying lumbar spine structures using ultrasound will vary by provider based on prior skills in ultrasound-guided procedures.⁸³ Thus, providers with prior ultrasound experience may require less training than those without such experience to achieve competency. However, extensive experience in performing landmark-guided LPs does not necessarily translate into rapid acquisition of skills to perform the procedure with ultrasound guidance. A study of practicing anesthesiologists with no prior ultrasound experience demonstrated that 20 supervised trials of ultrasound-guided spinal anesthesia were insufficient to achieve competency.⁸⁴ Although minimums may be a necessary step to gain competence, using them as a sole means to define competence does not account for variable learning curves.¹² Based on a national survey of 21 hospitalist procedure experts, the mean current vs suggested minimums for initial and ongoing hospital privileging for LPs were 1.8 vs 6.9 and 2.2 vs 4.6 annually in one report.⁸⁵

A fundamental question that needs to be answered is how to define competency in the use of ultrasound guidance for LP, including the specific skills and knowledge that must be mastered. At a minimum, providers must be able to identify lumbar spinous processes and distinguish them from the sacrum, identify the lumbar interspinous spaces and their corresponding levels, and estimate the depth from the skin to the ligamentum flavum from the midline and paramedian planes. Novice operators may benefit from practicing lumbar spine mapping of nonobese patients using a high-frequency linear transducer that generates high-resolution images and facilitates recognition of lumbar spine structures.

10) We recommend that novice providers should be supervised when performing ultrasound-guided LPs before performing the procedure independently on patients.

Rationale: Demonstration of competency in the use of ultrasound to identify lumbar spine anatomy should be achieved before routinely performing the procedure independently on patients.¹⁸ All providers will require a variable period of supervised practice to demonstrate the appropriate technique, followed by a period of unsupervised practice before competency is achieved. Supervised practice with guidance and feedback has been shown to significantly improve providers’ ability to delineate lumbar spine anatomy.⁸⁶

The process of producing these guidelines revealed areas of uncertainty and important gaps in the literature regarding the use of ultrasound guidance for LP.

First, it is unclear whether the use of ultrasound guidance for LP reduces postprocedural back pain and whether it improves patient satisfaction. Several studies have evaluated postprocedural back pain^{28,30,32,33,52} and patient satisfaction^28,29,33,51 with the use of ultrasound guidance, but these studies have found inconsistent results. Some of these results were probably due to insufficient statistical power or confounding variables. Furthermore, benefits have been demonstrated in certain subgroups, such as overweight patients or those with anatomical abnormalities, as was found in two studies.^52,87 Use of ultrasound guidance for spinal anesthesia has been shown to reduce postprocedural headache²⁸ and improve patient satisfaction⁵¹, although similar benefit has not been demonstrated in patients undergoing LP.

Second, the effect of using ultrasound guidance on the frequency of traumatic LPs is an area of uncertainty. A “traumatic tap” is defined as an inadvertent puncture of an epidural vein during passage of the spinal needle through the dura. It remains difficult to discern in these studies whether red blood cells detected in the CSF resulted from puncture of an epidural vein or from needle trauma of the skin and soft tissues. Despite this uncertainty, at least seven randomized controlled studies have assessed the effect of ultrasound guidance on traumatic LPs. The meta-analysis by Shaikh et al. included five randomized controlled studies that assessed the effect of ultrasound guidance on the reporting of traumatic taps. The study found a reduced risk of traumatic taps (risk ratio 0.27 [95% CI 0.11-0.67]), an absolute risk reduction of 5.9% (95% CI 2.3%-9.5%), and a number needed to treat of 17 (95% CI 11-44) to prevent one traumatic tap.¹⁶ Similarly, the meta-analysis by Gottlieb et al. showed a lower risk of traumatic taps among adults undergoing LP with ultrasound guidance in five randomized controlled studies with an odds ratio of 0.28 (95% CI 0.14-0.59). The meta-analysis by Gottlieb et al. included two adult studies that were not included by Shaikh et al.

Third, several important questions about the technique of ultrasound-guided LP remain unanswered. In addition to the static technique, a dynamic technique with real-time needle tracking has been described to perform ultrasound-guided LP, epidural catheterization, and spinal anesthesia. A pilot study by Grau et al. found that ultrasound used either statically or dynamically had fewer insertion attempts and needle redirections than use of landmarks alone.²⁹ Three other pilot studies showed successful spinal anesthesia in almost all patients^88-90 and one large study demonstrated successful spinal anesthesia with real-time ultrasound guidance in 97 of 100 patients with a median of three needle passes.⁹¹ Furthermore, a few industry-sponsored studies with small numbers of patients have described the use of novel needle tracking systems that facilitate needle visualization during real-time ultrasound-guided LP.^92,93 However, to our knowledge, no comparative studies of static versus dynamic guidance using novel needle tracking systems in human subjects have been published, and any potential role for these novel needle tracking systems has not yet been defined.

Finally, the effects of using ultrasound guidance on clinical decision-making, timeliness, and cost-effectiveness of LP have not yet been explored but could have important clinical practice implications.

Randomized controlled trials have demonstrated that using ultrasound guidance for LPs can reduce the number of needle insertion attempts and needle redirections and increase the overall procedural success rates. Ultrasound can more accurately identify the lumbar spine level than physical examination in both obese and nonobese patients, although the greatest benefit of using ultrasound guidance for LPs has been shown in obese patients.

Ultrasound permits assessment of the interspinous space width and measurement of the ligamentum flavum depth to select an optimal needle insertion site and adequate length spinal needle. Although the use of real-time ultrasound guidance has been described, the use of static ultrasound guidance for LP site marking remains the standard technique.

Acknowledgments

The authors thank all the members of the Society of Hospital Medicine Point-of-care Ultrasound Task Force and the Education Committee members for their time and dedication to develop these guidelines.

Collaborators from Society of Hospital Medicine Point-of-care Ultrasound Task Force: Saaid Abdel-Ghani, Robert Arntfield, Jeffrey Bates, Anjali Bhagra, Michael Blaivas, Daniel Brotman, Carolina Candotti, Richard Hoppmann, Susan Hunt, Trevor P. Jensen, Paul Mayo, Benji Mathews, Satyen Nichani, Vicki Noble, Martin Perez, Nitin Puri, Aliaksei Pustavoitau, Kreegan Reierson, Sophia Rodgers, Kirk Spencer, Vivek Tayal, David Tierney

SHM Point-of-care Ultrasound Task Force: CHAIRS: Nilam Soni, Ricardo Franco-Sadud, Jeff Bates. WORKING GROUPS: Thoracentesis Working Group: Ria Dancel (chair), Daniel Schnobrich, Nitin Puri. Vascular Access Working Group: Ricardo Franco (chair), Benji Matthews, Saaid Abdel-Ghani, Sophia Rodgers, Martin Perez, Daniel Schnobrich. Paracentesis Working Group: Joel Cho (chair), Benji Matthews, Kreegan Reierson, Anjali Bhagra, Trevor P. Jensen Lumbar Puncture Working Group: Nilam J. Soni (chair), Ricardo Franco, Gerard Salame, Josh Lenchus, Venkat Kalidindi, Ketino Kobaidze. Credentialing Working Group: Brian P Lucas (chair), David Tierney, Trevor P. Jensen PEER REVIEWERS: Robert Arntfield, Michael Blaivas, Richard Hoppmann, Paul Mayo, Vicki Noble, Aliaksei Pustavoitau, Kirk Spencer, Vivek Tayal. METHODOLOGIST: Mahmoud El Barbary. LIBRARIAN: Loretta Grikis. SOCIETY OF HOSPITAL MEDICINE EDUCATION COMMITTEE: Daniel Brotman (past chair), Satyen Nichani (current chair), Susan Hunt. SOCIETY OF HOSPITAL MEDICINE STAFF: Nick Marzano.

Disclosures

The authors have nothing to disclose.

Funding

Brian P Lucas: Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development and Dartmouth SYNERGY, National Institutes of Health, National Center for Translational Science (UL1TR001086). Nilam Soni: Department of Veterans Affairs, Quality Enhancement Research Initiative (QUERI) Partnered Evaluation Initiative Grant (HX002263-01A1).

Disclaimer

The contents of this publication do not represent the views of the U.S. Department of Veterans Affairs or the United States Government.

Approximately 400,000 lumbar punctures (LPs) are performed in the United States annually for either diagnostic workup or therapeutic relief.¹ Lumbar punctures are increasingly being performed in the United States, with an estimated 97,000 LPs performed on Medicare fee-for-service beneficiaries in 2011 alone, which is an increase of approximately 4,000 LPs in the same population from 1991.² Approximately 273,612 LPs were performed on hospitalized patients in the United States in 2010,¹ and the inpatient hospital setting is the most common site for LPs.^2,3

Many LPs are referred to radiologists who have access to imaging guidance to aid with needle insertion.² However, referrals to radiology delay performance of LPs, and delayed diagnosis of acute bacterial meningitis, the most common yet serious condition for which LPs are performed, is associated with increased morbidity and mortality.^4-8 Furthermore, although initiating empiric antibiotic treatment for suspected acute bacterial meningitis is recommended in some cases, doing so routinely can cause false-negative cerebrospinal fluid (CSF) culture results, complicating decisions about de-escalation and duration of antibiotics that could have been safely avoided by promptly performing an LP.⁹

Delaying the performance of LP has been associated with increased mortality.¹⁰ Demonstration of proficiency in performance of lumbar puncture is considered a core competency for hospitalists,¹¹ and with the increasing availability of point-of-care ultrasound, hospitalists can use ultrasound to guide performance of LPs at the bedside.¹² However, 30% of patients requiring LP in emergency departments have difficult-to-palpate lumbar spine landmarks,¹³ and lumbar puncture performed based on palpation of landmarks alone has been reported to fail or be traumatic in 28% of patients.¹⁴ Use of ultrasound guidance for lumbar puncture has been shown in randomized controlled trials to improve procedural success rates, while reducing the time to successful LP, needle passes, patient pain scores, and risk of a traumatic LP.^15-17

The purpose of this position statement is to review the literature and present consensus-based recommendations on the performance of ultrasound-guided LP in adult patients. This position statement does not mandate that hospitalists use ultrasound guidance for LP, nor does it establish ultrasound guidance as the standard of care for LP. Similar to previously published Society of Hospital Medicine (SHM) position statements,^12,18,19 this document presents recommendations with supporting evidence for the clinical outcomes, techniques, and training for using ultrasound guidance for LP. A manuscript describing the technique of ultrasound guidance for LPs has been previously published by some of the authors of this position statement.²⁰

Detailed methods are described in Appendix 1. The SHM Point-of-care Ultrasound (POCUS) Task Force was assembled to carry out this guideline development project under the direction of the SHM Board of Directors, Director of Education, and Education Committee. All expert panel members were physicians or advanced practice providers with expertise in POCUS. Expert panel members were divided into working group members, external peer reviewers, and a methodologist. All Task Force members were required to disclose any potential conflicts of interests (Appendix 2). The literature search was conducted in two independent phases. The first phase included literature searches conducted by the six working group members themselves. Key clinical questions and draft recommendations were then prepared. A systematic literature search was conducted by a medical librarian based on the findings of the initial literature search and draft recommendations. The Medline, Embase, CINAHL, and Cochrane medical databases were searched from 1975 to December 2015 initially. Google Scholar was also searched without limiters. Updated searches were conducted in November 2016, January 2018, and October 2018. The search strings are included in Appendix 3. All article abstracts were first screened for relevance by at least two members of the working group. Full-text versions of screened articles were reviewed, and articles on the use of ultrasound to guide LP were selected. In addition, the following article types were excluded: non-English language, nonhuman, age <18 years, meeting abstracts, meeting posters, narrative reviews, case reports, letters, and editorials. Moreover, studies focusing on the use of ultrasound guidance for spinal nerve root injections, regional anesthesia, and assessment of lumbar spine anatomy alone were excluded. All relevant systematic reviews, meta-analyses, randomized controlled trials, and observational studies of ultrasound-guided LP were screened and selected. Final article selection was based on working group consensus, and the selected literature was incorporated into the draft recommendations.

The Research and Development (RAND) Appropriateness Method that required panel judgment and consensus was used.²¹ The 27 voting members of the SHM POCUS Task Force reviewed and voted on the draft recommendations considering the following five transforming factors: (1) Problem priority and importance, (2) Level of quality of evidence, (3) Benefit/harm balance, (4) Benefit/burden balance, and (5) Certainty/concerns about PEAF (Preferences/Equity/Acceptability/Feasibility). Panel members participated in two rounds of electronic voting using an internet-based electronic data collection tool (REDCap™) in February 2018 and April 2018 (Appendix 4). Voting on appropriateness was conducted using a 9-point Likert scale. The three zones of the 9-point Likert scale were inappropriate (1-3 points), uncertain (4-6 points), and appropriate (7-9 points). The degree of consensus was assessed using the RAND algorithm (Appendix Figure 1 and Table 1). Establishing a recommendation required at least 70% agreement that a recommendation was “appropriate.” A strong recommendation required 80% of the votes within one integer of the median, following the RAND rules. Disagreement was defined as >30% of panelists voting outside of the zone of the median.

Recommendations were classified as strong or weak/conditional based on preset rules defining the panel’s level of consensus, which determined the wording of each recommendation (Table 2). The revised consensus-based recommendations underwent internal and external reviews by POCUS experts from different subspecialties. The final review of this position statement was performed by members of the SHM POCUS Task Force, SHM Education Committee, and SHM Executive Committee. The SHM Executive Committee endorsed this position statement in June 2018 before submission to the Journal of Hospital Medicine.

Literature Search

A total of 4,389 references were pooled from four different sources: a search by a certified medical librarian in December 2015 (3,212 citations) that was updated in November 2016 (380 citations), January 2018 (282 citations), and October 2018 (274 citations); working group members’ personal bibliographies and searches (31 citations); and a search focusing on ultrasound-guided LP training (210 citations). A total of 232 full-text articles were reviewed, and the final selection included 77 articles that were abstracted into a data table and incorporated into the draft recommendations. Details of the literature search strategy are presented in Appendix 3.

Four domains (clinical outcomes, technique, training, and knowledge gaps) with 16 draft recommendations were generated based on a review of the literature. Selected references were abstracted and assigned to each draft recommendation. Rationales for each recommendation were drafted citing supporting evidence. After two rounds of panel voting, five recommendations did not achieve agreement based on the RAND rules, one recommendation was combined with another recommendation during peer review, and 10 statements received final approval. The degree of consensus based on the median score and the dispersion of voting around the median are shown in Appendix 5. Nine statements were approved as strong recommendations, and one was approved as a conditional recommendation. Therefore, the final recommendation count was 10. The strength of the recommendation and degree of consensus for each recommendation are summarized in Table 1.

Terminology

LP is a procedure in which a spinal needle is introduced into the subarachnoid space for the purpose of collecting CSF for diagnostic evaluation and/or therapeutic relief.

Throughout this document, the phrases “ultrasound-guided” and “ultrasound guidance” refer to the use of ultrasound to mark a needle insertion site immediately before performing the procedure. This is also known as static ultrasound guidance. Real-time or dynamic ultrasound guidance refers to direct visualization of the needle tip as it traverses through the skin and soft tissues to reach the ligamentum flavum. Any reference to real-time ultrasound guidance is explicitly stated.

Clinical outcomes

1) When ultrasound equipment is available, along with providers who are appropriately trained to use it, we recommend that ultrasound guidance should be used for site selection of LPs to reduce the number of needle insertion attempts and needle redirections and increase the overall procedure success rates, especially in patients who are obese or have difficult-to-palpate landmarks.

Rationale. LPs have historically been performed by selecting a needle insertion site based on palpation of anatomical landmarks. However, an estimated 30% of patients requiring LP in emergency departments have lumbar spine landmarks that are difficult to palpate, most commonly due to obesity.¹³ Furthermore, lumbar puncture performed based on palpation of landmarks alone has been reported to fail in 28% of patients.¹⁴

Ultrasound can be used at the bedside to elucidate the lumbar spine anatomy to guide performance of LP or epidural catheterization. Since the early 2000s, randomized studies comparing the use of ultrasound guidance (ultrasound-guided) versus anatomical landmarks (landmark-guided) to map the lumbar spine for epidural catheterization have emerged. It is important to recognize that the exact same ultrasound technique is used for site marking of LP, epidural catheterization, and spinal anesthesia—the key difference is how deep the needle tip is inserted. Therefore, data from these three ultrasound-guided procedures are often pooled. Currently, at least 33 randomized controlled studies comparing ultrasound-guided vs landmark-guided site selection for LP, epidural catheterization, or spinal anesthesia have been published.^22-49 We present three meta-analyses below that pooled data primarily from randomized controlled studies comparing ultrasound-guided vs landmark-guided site selection for LP or spinal anesthesia.

In 2013, Shaikh et al. published the first meta-analysis with 14 randomized controlled studies comparing ultrasound-guided vs landmark-guided site selection for LP (n = 5) or epidural catheterization (n = 9). The pooled data showed that use of ultrasound guidance decreased the proportion of failed procedures (risk ratio 0.21, 95% CI 0.10-0.43) with an absolute risk reduction of 6.3% (95% CI 4.1%-8.4%) and a number needed to treat of 16 (95% CI 12-25) to prevent one failed procedure. In addition, the use of ultrasound reduced the mean number of attempts by 0.44 (95% CI 0.24-0.64) and reduced the mean number of needle redirections by 1.00 (95% CI 0.75-1.24). The reduction in risk of a failed procedure was similar for LPs (risk ratio 0.19 [95% CI 0.07-0.56]) and epidural catheterizations (risk ratio 0.23 [95% CI 0.09-0.60]).¹⁶

A similar meta-analysis published by Perlas et al. in 2016 included a total of 31 studies, both randomized controlled and cohort studies, evaluating the use of ultrasound guidance for LP, spinal anesthesia, and epidural catheterization.⁵⁰ The goal of this systematic review and meta-analysis was to establish clinical practice recommendations. The authors concluded (1) the data consistently suggest that ultrasound is more accurate than palpation for lumbar interspace identification, (2) ultrasound allows accurate measurement of the needle insertion depth to reach the epidural space with a mean difference of <3 mm compared with the actual needle insertion depth, and (3) ultrasound increases the efficacy of lumbar epidural or spinal anesthesia by decreasing the mean number of needle passes for success by 0.75 (95% CI 0.44-1.07) and reducing the risk of a failed procedure (risk ratio 0.51 [95% CI 0.32-0.80]), both in patients with normal surface anatomy and in those with technically difficult surface anatomy due to obesity, scoliosis, or previous spine surgery.

Compared to the two earlier meta-analyses that included studies of both LP and spinal anesthesia procedures, the meta-analysis conducted by Gottlieb et al. in 2018 pooled data from 12 randomized controlled studies of ultrasound guidance for LPs only. For the primary outcome, pooled data from both adult and pediatric studies demonstrated higher procedural success rates with ultrasound-guided vs landmark-guided LPs (90% vs 81%) with an odds ratio of 2.1 (95% CI 0.66-7.44) in favor of ultrasound; however, there were no statistically significant differences when the adult and pediatric subgroups were analyzed separately, probably due to underpowering. For the secondary outcomes, data from the adult subgroup showed that use of ultrasound guidance was associated with fewer traumatic LPs (OR 0.28, 95% CI 0.14-0.59), shorter time to procedural success (adjusted mean difference –3.03 minutes, 95% CI –3.54 to –2.52), fewer number of needle passes (adjusted mean difference –0.81 passes, 95% CI –1.57 to –0.05), and lower patient pain scores (adjusted mean difference –2.53, 95% CI –3.89 to –1.17).

At least 12 randomized controlled studies have been published comparing the use of ultrasound guidance vs landmarks for the performance of LP or spinal anesthesia in adult patients, which were not included in the abovementioned meta-analyses. These individual studies demonstrated similar benefits of using ultrasound guidance: reduced needle insertion attempts, reduced needle redirections, and increased overall procedural success rates.^{17,31,37,40,41,43-49}

It is important to recognize that four randomized controlled studies did not demonstrate any benefits of ultrasound guidance on the number of attempts or procedural success rates,^23,33,41,51 and three of these studies were included in the abovementioned meta-analyses.^23,33,51 Limitations of these negative studies include potential selection bias, inadequate sample sizes, and varying levels of operator skills in procedures, ultrasound guidance, or both. One study included emergency medicine residents as operators with varying degrees of ultrasound skills, and more importantly, patient enrollment occurred by convenience sampling, which may have introduced selection bias. Furthermore, most of the patients were not obese (median BMI of 27 kg/m²), and it is unclear why 10 years lapsed from data collection until publication.³³ Another study with three experienced anesthesiologists as operators performing spinal anesthesia enrolled only patients who were not obese (mean BMI of 29 kg/m²) and had easily palpable bony landmarks—two patient characteristics associated with the least benefit of using ultrasound guidance in other studies.²³ Another negative study had one experienced anesthesiologist marking obstetric patients with ultrasound, but junior residents performing the actual procedure in the absence of the anesthesiologist who had marked the patient.⁴¹

In general, the greatest benefit of using ultrasound guidance for LP has been demonstrated in obese patients.^{24,32,34,35,52,53} Benefits have been shown in specific obese patient populations, including obstetric,^31,54,55 orthopedic,^24,56,57 and emergency department patients.³⁰

By increasing the procedural success rates with the use of ultrasound at the bedside, fewer patients may be referred to interventional radiology for fluoroscopic-guided LP, decreasing the patient exposure to ionizing radiation. A randomized study (n = 112) that compared site marking with ultrasound guidance versus fluoroscopic guidance for epidural steroid injections found the two techniques to be equivalent with respect to mean procedure time, number of needle insertion attempts, or needle passes.⁵⁸ Another randomized study found that the performance time of ultrasound guidance was two minutes shorter (P < .05) than fluoroscopic guidance.⁵⁹

Techniques

2) We recommend that ultrasound should be used to more accurately identify the lumbar spine level than physical examination in both obese and nonobese patients.

Rationale. Traditionally, an imaginary line connecting the iliac crests (intercristal line, Tuffier’s line, or Jacoby’s line) was considered to identify the L4 vertebra or the L4-L5 interspinous space in the midline; however, studies have revealed this traditional landmark to be much less accurate than previously thought. In general, palpating the iliac crests to mark the intercristal line identifies an interspinous space that is one space cephalad (ie, the L2-L3 interspinous space) but can range from L1-L2 to L4-L5.^46,60-64 If an LP is inadvertently performed in the L1-L2 interspinous space, the risk of spinal cord injury is higher than that when performed in a more distal interspinous space.

A study by Margarido et al. with 45 patients with a mean BMI of 30 kg/m² found that the intercristal line was located above the L4-L5 interspinous space in 100% of patients. More importantly, the intercristal line was above L2-L3 in 36% of patients and above L1-L2 in 4% of patients. It is important to note that patients with scoliosis or previous spine surgery were excluded from this study, and all examinations were performed by two experienced anesthesiologists with patients in a sitting position—all factors that would favor accurate palpation and marking of the iliac crests.⁶⁰

In a study of nonobese patients (mean BMI 28 kg/m²) undergoing spinal anesthesia, Duniec et al. compared the lumbar level identified by palpation versus ultrasound and found discordance between the two techniques in 36% of patients; 18% were one space too cephalad, 16% were one space too caudal, and 2% were off by two interspinous spaces.⁶¹ Another study found discordance in 64% of patients (mean BMI 28 kg/m²) when comparing the interspinous level where spinal anesthesia had been performed by palpation versus a post-procedural ultrasound examination. This study revealed that the interspinous space was more cephalad in 50% of patients with 6% of punctures performed in the L1-L2 interspace.⁶² A similar study compared the accuracy of palpation vs ultrasound to identify the L3-L4 interspinous space in obese (mean BMI 34 kg/m²) versus nonobese (mean BMI 27 kg/m²) patients. This study found marking a space above L3-L4 in 51% of obese and 40% of nonobese patients and marking of the L1-L2 interspace in 7% of obese and 4% of nonobese patients.⁶⁴

A study comparing palpation vs ultrasound found that 68% of obese patients with a BMI of >30 kg/m² had difficult-to-palpate lumbar spine landmarks, but with the use of ultrasound, landmarks were identified in 76% of all patients, including obese and nonobese, with difficult-to-palpate landmarks.⁶⁵

3) We suggest using ultrasound for selecting and marking a needle insertion site just before performing LPs in either a lateral decubitus or sitting position. The patient should remain in the same position after marking the needle insertion site.

Rationale. Ultrasound mapping of the lumbar spine can be performed in either a lateral decubitus or sitting position. Selecting and marking a needle insertion site should be performed at the bedside just before performing the procedure. The patient must remain in the same position in the interim between marking and inserting the needle, as a slight change in position can alter the needle trajectory, lowering the LP success rate. Although performing LPs in a lateral decubitus position has the advantage of accurately measuring the opening pressure, misalignment of the shoulder and pelvic girdles and bowing of the bed in a lateral decubitus position may lower LP success rates.

One randomized study comparing ultrasound-guided spinal anesthesia in a lateral decubitus versus sitting position found no difference in the number of needle insertion attempts or measurement of the skin-dura distance; however, the needle insertion depth was 0.73 cm greater in a lateral decubitus vs sitting position (P = .002).⁶⁶ Procedural success rates of LP with ultrasound guidance have not been directly compared in a sitting versus lateral decubitus position, although the overall procedural success rates were higher in one study that allowed the operator to choose either sitting or lateral decubitus position when ultrasound was used.³²

4) We recommend that a low-frequency transducer, preferably a curvilinear array transducer, should be used to evaluate the lumbar spine and mark a needle insertion site in most patients. A high-frequency linear array transducer may be used in nonobese patients.

Rationale. Low-frequency transducers emit sound waves that penetrate deep tissues, allowing visualization of bones and ligaments of the lumbar spine. A high-frequency linear transducer offers better resolution but shallower penetration to approximately 6-9 cm, limiting its use for site marking in overweight and obese patients. In obese patients, the ligamentum flavum is often deeper than 6 cm, which requires a low-frequency transducer to be visualized.

Most of the randomized controlled studies demonstrating benefits of using ultrasound guidance compared with landmark guidance for performance of LP, epidural anesthesia, or spinal anesthesia have used a low-frequency, curvilinear transducer.^{22,24,26-28,31,34-36,39,43-45,67} Two randomized controlled trials used a high-frequency linear transducer for site marking of lumbar procedures.^30,32,37 Using a high-frequency linear transducer has been described in real-time, ultrasound-guided LPs, the advantage being better needle visualization with a linear transducer.²⁹ Detection of blood vessels by color flow Doppler may be another advantage of using a high-frequency linear transducer, although a study by Grau et al. showed that use of color flow Doppler with a low-frequency curvilinear transducer permitted visualization of interspinous vessels as small as 0.5 mm in size.⁶⁸

5) We recommend that ultrasound should be used to map the lumbar spine, starting at the level of the sacrum and sliding the transducer cephalad, sequentially identifying the lumbar spine interspaces.Rationale. Although no studies have directly compared different ultrasound scanning protocols to map the lumbar spine, starting at the level of the sacrum and sliding the transducer cephalad to sequentially identify the lumbar interspinous spaces is the most commonly described technique in studies demonstrating improved clinical outcomes with the use of ultrasound.^{24,31,34,37,39,40,45,56,57,67} Because the sacrum can be easily recognized, identifying it first is most beneficial in patients with few or no palpable landmarks.

All five lumbar spinous processes and interspinous spaces can be mapped from the sacrum using either a midline or a paramedian approach, and the widest interspinous space can be selected. In a midline approach, either a transverse or a longitudinal view is obtained. The transducer is centered on the sacrum and slid cephalad from L5 to L1 to identify each spinous process and interspinous space. In a paramedian approach, longitudinal paramedian views are obtained from the L5–sacrum interspace to the L1–L2 interspace, and each interspinous space is identified as the transducer is slid cephalad. Both these approaches are effective for mapping the lumbar spine. Whether the entire lumbar spine is mapped, and whether a midline or a paramedian approach is utilized, will depend on the operator’s preference.

6) We recommend that ultrasound should be used in a transverse plane to mark the midline of the lumbar spine and a longitudinal plane to mark the interspinous spaces. The intersection of these two lines marks the needle insertion site.

Rationale. The most common technique described in comparative studies of ultrasound vs landmarks includes visualization of the lumbar spine in two planes, a transverse plane to identify the midline and a longitudinal plane to identify the interspinous spaces. The majority of randomized controlled studies that demonstrated a reduction in the number of needle insertion attempts and an increase in the procedural success rates have used this technique (see Clinical Outcomes).^{22,24,28,32,35-37,43,44} Marking the midline and interspinous space(s) for LP may be performed in any order, starting with either the transverse or longitudinal plane first.

The midline of the spine is marked by placing the transducer in a transverse plane over the lumbar spine, centering over the spinous processes that have a distinct hyperechoic tip and a prominent acoustic shadow deep to the bone, and drawing a line perpendicular to the center of the transducer delineating the midline. The midline should be marked over a minimum of two or three spinous processes.

To identify the interspinous spaces, the transducer is aligned longitudinally over the midline. The transducer is slid along the midline to identify the widest interspinous space. Once the transducer is centered over the widest interspinous space, a line perpendicular to the center of the transducer is drawn to mark the interspinous space. The intersection of the lines marking the spinal midline and the selected interspinous space identifies the needle entry point.

To visualize the ligamentum flavum from a paramedian view, the transducer is oriented longitudinally over the midline, slid approximately 1 cm laterally, and tilted approximately 15 degrees aiming the ultrasound beam toward the midline. The skin–ligamentum flavum distance is most reliably measured from a paramedian view. Alternatively, in some patients, the ligamentum flavum may be visualized in the midline and the depth can be measured.

7) We recommend that ultrasound should be used during a preprocedural evaluation to measure the distance from the skin surface to the ligamentum flavum from a longitudinal paramedian view to estimate the needle insertion depth and ensure that a spinal needle of adequate length is used.

Rationale. The distance from the skin to the ligamentum flavum can be measured using ultrasound during preprocedural planning. Knowing the depth to the ligamentum flavum preprocedurally allows the operator to procure a spinal needle of adequate length, anticipate the insertion depth before CSF can be obtained, determine the depth to which a local anesthetic will need to be injected, and decide whether the anticipated difficulty of the procedure warrants referral to or consultation with another specialist.

The skin–ligamentum flavum distance can be measured from a transverse midline view or a longitudinal paramedian view. A longitudinal paramedian view provides an unobstructed view of the ligamentum flavum due to less shadowing from bony structures compared with a midline view. Several studies have demonstrated a strong correlation between the skin–ligamentum flavum distance measured by ultrasound and the actual needle insertion depth in both midline and paramedian views.^{28,34,36,53,54,57,69,70}

A meta-analysis that included 13 comparative studies evaluating the correlation between ultrasound-measured depth and actual needle insertion depth to reach the epidural or intrathecal space consistently demonstrated a strong correlation between the measured and actual depth.⁵⁰ A few studies have reported near-perfect Pearson correlation coefficients of 0.98.^55,71,72 The pooled correlation was 0.91 (95% CI 0.87-0.94). All studies measured the depth from the skin to the ventral side of the ligamentum flavum or the intrathecal space from either a longitudinal paramedian view (n = 4) or a transverse midline view (n = 9). Eight of the more recent studies evaluated the accuracy of the ultrasound measurements and found the depth measurements by ultrasound to be accurate within 1-13 mm of the actual needle insertion depth, with seven of the eight studies reporting a mean difference of ≤3 mm.⁵⁰

Measurement of the distance between the skin and the ligamentum flavum generally underestimates the needle insertion depth. One study reported that measurement of the skin–ligamentum flavum distance underestimates the needle insertion depth by 7.6 mm to obtain CSF, whereas measurement of the skin–posterior longitudinal ligament distance overestimates the needle insertion depth by 2.5 mm.⁵⁷ A well-accepted contributor to underestimation of the depth measurements using ultrasound is compression of the skin and soft tissues by the transducer, and therefore, pressure on the skin must be released before freezing an image and measuring the depth to the subarachnoid space.

Training

8) We recommend that novices should undergo simulation-based training, where available, before attempting ultrasound-guided LPs on actual patients.

Rationale. Similar to training for other bedside procedures, dedicated training sessions, including didactics, supervised practice on patients, and simulation-based practice, should be considered when teaching novices to perform ultrasound-guided LP. Simulation-based training facilitates acquisition of knowledge and skills to perform invasive bedside procedures, including LP.⁷³ Simulation-based training has been commonly incorporated into procedure training for trainees using an immersive experience, such as a “boot camp,”^74-77 or a standardized curriculum,^78,79 and has demonstrated improvements in post-course procedural knowledge, technical skills, and operator confidence. Two of these studies included training in the use of ultrasound guidance for LP. These studies showed that simulation-based practice improved skill acquisition and confidence.^80,81 Simulation using novel computer software may improve skill acquisition in the use of ultrasound guidance for LP.⁸²

9) We recommend that training in ultrasound-guided LPs should be adapted based on prior ultrasound experience, as learning curves will vary.Rationale. The learning curve to achieve competency in the use of ultrasound guidance for LP has not been well studied. The rate of attaining competency in identifying lumbar spine structures using ultrasound will vary by provider based on prior skills in ultrasound-guided procedures.⁸³ Thus, providers with prior ultrasound experience may require less training than those without such experience to achieve competency. However, extensive experience in performing landmark-guided LPs does not necessarily translate into rapid acquisition of skills to perform the procedure with ultrasound guidance. A study of practicing anesthesiologists with no prior ultrasound experience demonstrated that 20 supervised trials of ultrasound-guided spinal anesthesia were insufficient to achieve competency.⁸⁴ Although minimums may be a necessary step to gain competence, using them as a sole means to define competence does not account for variable learning curves.¹² Based on a national survey of 21 hospitalist procedure experts, the mean current vs suggested minimums for initial and ongoing hospital privileging for LPs were 1.8 vs 6.9 and 2.2 vs 4.6 annually in one report.⁸⁵

A fundamental question that needs to be answered is how to define competency in the use of ultrasound guidance for LP, including the specific skills and knowledge that must be mastered. At a minimum, providers must be able to identify lumbar spinous processes and distinguish them from the sacrum, identify the lumbar interspinous spaces and their corresponding levels, and estimate the depth from the skin to the ligamentum flavum from the midline and paramedian planes. Novice operators may benefit from practicing lumbar spine mapping of nonobese patients using a high-frequency linear transducer that generates high-resolution images and facilitates recognition of lumbar spine structures.

10) We recommend that novice providers should be supervised when performing ultrasound-guided LPs before performing the procedure independently on patients.

Rationale: Demonstration of competency in the use of ultrasound to identify lumbar spine anatomy should be achieved before routinely performing the procedure independently on patients.¹⁸ All providers will require a variable period of supervised practice to demonstrate the appropriate technique, followed by a period of unsupervised practice before competency is achieved. Supervised practice with guidance and feedback has been shown to significantly improve providers’ ability to delineate lumbar spine anatomy.⁸⁶

The process of producing these guidelines revealed areas of uncertainty and important gaps in the literature regarding the use of ultrasound guidance for LP.

First, it is unclear whether the use of ultrasound guidance for LP reduces postprocedural back pain and whether it improves patient satisfaction. Several studies have evaluated postprocedural back pain^{28,30,32,33,52} and patient satisfaction^28,29,33,51 with the use of ultrasound guidance, but these studies have found inconsistent results. Some of these results were probably due to insufficient statistical power or confounding variables. Furthermore, benefits have been demonstrated in certain subgroups, such as overweight patients or those with anatomical abnormalities, as was found in two studies.^52,87 Use of ultrasound guidance for spinal anesthesia has been shown to reduce postprocedural headache²⁸ and improve patient satisfaction⁵¹, although similar benefit has not been demonstrated in patients undergoing LP.

Second, the effect of using ultrasound guidance on the frequency of traumatic LPs is an area of uncertainty. A “traumatic tap” is defined as an inadvertent puncture of an epidural vein during passage of the spinal needle through the dura. It remains difficult to discern in these studies whether red blood cells detected in the CSF resulted from puncture of an epidural vein or from needle trauma of the skin and soft tissues. Despite this uncertainty, at least seven randomized controlled studies have assessed the effect of ultrasound guidance on traumatic LPs. The meta-analysis by Shaikh et al. included five randomized controlled studies that assessed the effect of ultrasound guidance on the reporting of traumatic taps. The study found a reduced risk of traumatic taps (risk ratio 0.27 [95% CI 0.11-0.67]), an absolute risk reduction of 5.9% (95% CI 2.3%-9.5%), and a number needed to treat of 17 (95% CI 11-44) to prevent one traumatic tap.¹⁶ Similarly, the meta-analysis by Gottlieb et al. showed a lower risk of traumatic taps among adults undergoing LP with ultrasound guidance in five randomized controlled studies with an odds ratio of 0.28 (95% CI 0.14-0.59). The meta-analysis by Gottlieb et al. included two adult studies that were not included by Shaikh et al.

Third, several important questions about the technique of ultrasound-guided LP remain unanswered. In addition to the static technique, a dynamic technique with real-time needle tracking has been described to perform ultrasound-guided LP, epidural catheterization, and spinal anesthesia. A pilot study by Grau et al. found that ultrasound used either statically or dynamically had fewer insertion attempts and needle redirections than use of landmarks alone.²⁹ Three other pilot studies showed successful spinal anesthesia in almost all patients^88-90 and one large study demonstrated successful spinal anesthesia with real-time ultrasound guidance in 97 of 100 patients with a median of three needle passes.⁹¹ Furthermore, a few industry-sponsored studies with small numbers of patients have described the use of novel needle tracking systems that facilitate needle visualization during real-time ultrasound-guided LP.^92,93 However, to our knowledge, no comparative studies of static versus dynamic guidance using novel needle tracking systems in human subjects have been published, and any potential role for these novel needle tracking systems has not yet been defined.

Finally, the effects of using ultrasound guidance on clinical decision-making, timeliness, and cost-effectiveness of LP have not yet been explored but could have important clinical practice implications.

Randomized controlled trials have demonstrated that using ultrasound guidance for LPs can reduce the number of needle insertion attempts and needle redirections and increase the overall procedural success rates. Ultrasound can more accurately identify the lumbar spine level than physical examination in both obese and nonobese patients, although the greatest benefit of using ultrasound guidance for LPs has been shown in obese patients.

Ultrasound permits assessment of the interspinous space width and measurement of the ligamentum flavum depth to select an optimal needle insertion site and adequate length spinal needle. Although the use of real-time ultrasound guidance has been described, the use of static ultrasound guidance for LP site marking remains the standard technique.

Acknowledgments

The authors thank all the members of the Society of Hospital Medicine Point-of-care Ultrasound Task Force and the Education Committee members for their time and dedication to develop these guidelines.

Collaborators from Society of Hospital Medicine Point-of-care Ultrasound Task Force: Saaid Abdel-Ghani, Robert Arntfield, Jeffrey Bates, Anjali Bhagra, Michael Blaivas, Daniel Brotman, Carolina Candotti, Richard Hoppmann, Susan Hunt, Trevor P. Jensen, Paul Mayo, Benji Mathews, Satyen Nichani, Vicki Noble, Martin Perez, Nitin Puri, Aliaksei Pustavoitau, Kreegan Reierson, Sophia Rodgers, Kirk Spencer, Vivek Tayal, David Tierney

SHM Point-of-care Ultrasound Task Force: CHAIRS: Nilam Soni, Ricardo Franco-Sadud, Jeff Bates. WORKING GROUPS: Thoracentesis Working Group: Ria Dancel (chair), Daniel Schnobrich, Nitin Puri. Vascular Access Working Group: Ricardo Franco (chair), Benji Matthews, Saaid Abdel-Ghani, Sophia Rodgers, Martin Perez, Daniel Schnobrich. Paracentesis Working Group: Joel Cho (chair), Benji Matthews, Kreegan Reierson, Anjali Bhagra, Trevor P. Jensen Lumbar Puncture Working Group: Nilam J. Soni (chair), Ricardo Franco, Gerard Salame, Josh Lenchus, Venkat Kalidindi, Ketino Kobaidze. Credentialing Working Group: Brian P Lucas (chair), David Tierney, Trevor P. Jensen PEER REVIEWERS: Robert Arntfield, Michael Blaivas, Richard Hoppmann, Paul Mayo, Vicki Noble, Aliaksei Pustavoitau, Kirk Spencer, Vivek Tayal. METHODOLOGIST: Mahmoud El Barbary. LIBRARIAN: Loretta Grikis. SOCIETY OF HOSPITAL MEDICINE EDUCATION COMMITTEE: Daniel Brotman (past chair), Satyen Nichani (current chair), Susan Hunt. SOCIETY OF HOSPITAL MEDICINE STAFF: Nick Marzano.

Disclosures

The authors have nothing to disclose.

Funding

Brian P Lucas: Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development and Dartmouth SYNERGY, National Institutes of Health, National Center for Translational Science (UL1TR001086). Nilam Soni: Department of Veterans Affairs, Quality Enhancement Research Initiative (QUERI) Partnered Evaluation Initiative Grant (HX002263-01A1).

Disclaimer

The contents of this publication do not represent the views of the U.S. Department of Veterans Affairs or the United States Government.

Recognizing the increasingly important role of point-of-care ultrasound (POCUS) in advancing clinical care, the Society of Hospital Medicine (SHM) has published a valuable series of position statements to guide hospitalists and administrators on the safe and effective use of POCUS.¹ In this issue of the Journal of Hospital Medicine, Soni et al. present a series of consensus-based recommendations on ultrasound guidance for lumbar puncture (LP).² Among these are the recommendations that ultrasound “should be used” to map the lumbar spine and to select an appropriate puncture site to reduce insertion attempts, reduce needle redirections, and increase overall procedural success.

At first glance, the recommendations appear definitive. However, not immediately obvious is the authors’ clarification that “This position statement does not mandate that hospitalists use ultrasound guidance for LP, nor does it establish ultrasound guidance as the standard of care for LP.” Even with the authors’ caveat, this nuance may not be readily apparent to the readers who review only the executive summary of the guidelines or who omit the context provided in the background of the position statement.

The directive language of this position statement may be a result of an unmerited amplification. The SHM POCUS Task Force employed the Research and Development Appropriateness Method to quantify the degree of consensus and the strength of the recommendation assigned,³ reaching “very good” consensus for each of the recommendations espoused in its position statement. Procedurally, this implies that ≥80% of the 27 voting members rated each published recommendation statement as “appropriate”. Using wording assigned a priori by the committee to each level of consensus, appropriateness became magnified to the declaration “should be used”. In this manner, the strength of the recommendations in this position statement is not necessarily based on the experts’ convictions related to ultrasound-guided LP, nor the strength of the supporting evidence.

In the case of ultrasound-guided LP, we might choose different descriptors than “appropriate” or “should be used”. The evidence base for ultrasound guidance for LP, though growing, may be insufficient as a foundation to a position statement and is certainly insufficient to create a new standard of care for hospitalists. Although the SHM POCUS Task Force completed a thoughtful literature review, no systematic approach (eg, GRADE methodology⁴) was used to rate the quality of evidence. Furthermore, the literature reviewed was drawn predominantly from anesthesia and emergency medicine sources—not readily generalizable to the hospitalist. Notably, these studies examined all neuraxial procedures (most commonly epidural and spinal anesthesia), which employ different techniques and tools than LP and are performed by clinicians with vastly different procedural training backgrounds than most hospitalists. Altogether, this creates the potential for a gap between true evidence quality and the strength of recommendation.

At a high level, although the technique for ultrasound mapping of the lumbar spine may be similar, the use of ultrasound has been less well studied specifically for LP. When considering LP alone, the available literature is inadequate to recommend uniform ultrasound guidance. A 2018 meta-analysis by Gottlieb et al. included 12 studies focusing only on LP, totaling N = 957 patients.⁵ This showed some favorability of ultrasound guidance, with a success rate of 90% using ultrasound, 81.4% with a landmark-based approach, and an odds ratio of 2.22 favoring ultrasound guidance (95% CI: 1.03-4.77). Unfortunately, when focusing only on adult patients, the advantage of POCUS diminished, with 91.4% success in the ultrasound group, 87.7% success in the landmark group, and a nonsignificant odds ratio of 2.10 (95% CI: 0.66-7.44).

Unequivocally, POCUS has established itself as a transformative technology for the guidance of invasive bedside procedures, bringing increased procedural success, improved safety, and decreased complication rates.⁶ For some procedures, particularly central venous catheterization, ultrasound guidance is a clear standard of care.^7,8 For LP, the greatest benefit has been observed in patients with anticipated procedural challenges, most commonly obese patients in whom landmarks are not easily palpable.⁹ Moreover, the harms ultrasound seeks to prevent are substantially different. The primary risk of deferring ultrasound guidance for LP is most often a failed procedure, whereas for other common ultrasound-guided procedures, the harms may include significant vascular injury, pneumothorax, or bowel perforation. Differences in the relative harms make risk-benefit assessments harder to quantify and studies harder to carry out.

Sonographic guidance for LP has a role in clinical practice and should always be considered. However, at present, there exist no guidelines in any other specialty regarding the routine use of ultrasound-guided LP, including anesthesia, emergency medicine, neurology, or interventional radiology.^10-15 As a result, a conservative interpretation of the POCUS Task Force’s findings would be to consider the use of ultrasound guidance for LP in patients where landmark identification is particularly challenging, but not to consider it a standard requirement for accreditation, training, or practice as of yet. Saying “more studies are required” can be a cop-out in some cases, but in this situation, the old adage does seem to apply.

We have great respect for the work of the SHM POCUS Task Force in advancing the use of POCUS in hospital medicine. Though ultrasound is not currently mandated as a care standard for the performance of LP, we all can agree that POCUS does confer advantages for this procedure, particularly in a well-selected patient population. To continue to provide care of the highest quality, hospitalists must be encouraged to elevate their practice with POCUS and be supported with the equipment, training, credentialing, and quality assurance structures necessary to integrate bedside ultrasound safely and effectively into their diagnostic and procedural practice.

Disclosures

No conflicts of interest to disclose.

Funding

None.

Recognizing the increasingly important role of point-of-care ultrasound (POCUS) in advancing clinical care, the Society of Hospital Medicine (SHM) has published a valuable series of position statements to guide hospitalists and administrators on the safe and effective use of POCUS.¹ In this issue of the Journal of Hospital Medicine, Soni et al. present a series of consensus-based recommendations on ultrasound guidance for lumbar puncture (LP).² Among these are the recommendations that ultrasound “should be used” to map the lumbar spine and to select an appropriate puncture site to reduce insertion attempts, reduce needle redirections, and increase overall procedural success.

At first glance, the recommendations appear definitive. However, not immediately obvious is the authors’ clarification that “This position statement does not mandate that hospitalists use ultrasound guidance for LP, nor does it establish ultrasound guidance as the standard of care for LP.” Even with the authors’ caveat, this nuance may not be readily apparent to the readers who review only the executive summary of the guidelines or who omit the context provided in the background of the position statement.

The directive language of this position statement may be a result of an unmerited amplification. The SHM POCUS Task Force employed the Research and Development Appropriateness Method to quantify the degree of consensus and the strength of the recommendation assigned,³ reaching “very good” consensus for each of the recommendations espoused in its position statement. Procedurally, this implies that ≥80% of the 27 voting members rated each published recommendation statement as “appropriate”. Using wording assigned a priori by the committee to each level of consensus, appropriateness became magnified to the declaration “should be used”. In this manner, the strength of the recommendations in this position statement is not necessarily based on the experts’ convictions related to ultrasound-guided LP, nor the strength of the supporting evidence.

In the case of ultrasound-guided LP, we might choose different descriptors than “appropriate” or “should be used”. The evidence base for ultrasound guidance for LP, though growing, may be insufficient as a foundation to a position statement and is certainly insufficient to create a new standard of care for hospitalists. Although the SHM POCUS Task Force completed a thoughtful literature review, no systematic approach (eg, GRADE methodology⁴) was used to rate the quality of evidence. Furthermore, the literature reviewed was drawn predominantly from anesthesia and emergency medicine sources—not readily generalizable to the hospitalist. Notably, these studies examined all neuraxial procedures (most commonly epidural and spinal anesthesia), which employ different techniques and tools than LP and are performed by clinicians with vastly different procedural training backgrounds than most hospitalists. Altogether, this creates the potential for a gap between true evidence quality and the strength of recommendation.

At a high level, although the technique for ultrasound mapping of the lumbar spine may be similar, the use of ultrasound has been less well studied specifically for LP. When considering LP alone, the available literature is inadequate to recommend uniform ultrasound guidance. A 2018 meta-analysis by Gottlieb et al. included 12 studies focusing only on LP, totaling N = 957 patients.⁵ This showed some favorability of ultrasound guidance, with a success rate of 90% using ultrasound, 81.4% with a landmark-based approach, and an odds ratio of 2.22 favoring ultrasound guidance (95% CI: 1.03-4.77). Unfortunately, when focusing only on adult patients, the advantage of POCUS diminished, with 91.4% success in the ultrasound group, 87.7% success in the landmark group, and a nonsignificant odds ratio of 2.10 (95% CI: 0.66-7.44).

Unequivocally, POCUS has established itself as a transformative technology for the guidance of invasive bedside procedures, bringing increased procedural success, improved safety, and decreased complication rates.⁶ For some procedures, particularly central venous catheterization, ultrasound guidance is a clear standard of care.^7,8 For LP, the greatest benefit has been observed in patients with anticipated procedural challenges, most commonly obese patients in whom landmarks are not easily palpable.⁹ Moreover, the harms ultrasound seeks to prevent are substantially different. The primary risk of deferring ultrasound guidance for LP is most often a failed procedure, whereas for other common ultrasound-guided procedures, the harms may include significant vascular injury, pneumothorax, or bowel perforation. Differences in the relative harms make risk-benefit assessments harder to quantify and studies harder to carry out.

Sonographic guidance for LP has a role in clinical practice and should always be considered. However, at present, there exist no guidelines in any other specialty regarding the routine use of ultrasound-guided LP, including anesthesia, emergency medicine, neurology, or interventional radiology.^10-15 As a result, a conservative interpretation of the POCUS Task Force’s findings would be to consider the use of ultrasound guidance for LP in patients where landmark identification is particularly challenging, but not to consider it a standard requirement for accreditation, training, or practice as of yet. Saying “more studies are required” can be a cop-out in some cases, but in this situation, the old adage does seem to apply.

We have great respect for the work of the SHM POCUS Task Force in advancing the use of POCUS in hospital medicine. Though ultrasound is not currently mandated as a care standard for the performance of LP, we all can agree that POCUS does confer advantages for this procedure, particularly in a well-selected patient population. To continue to provide care of the highest quality, hospitalists must be encouraged to elevate their practice with POCUS and be supported with the equipment, training, credentialing, and quality assurance structures necessary to integrate bedside ultrasound safely and effectively into their diagnostic and procedural practice.

Disclosures

No conflicts of interest to disclose.

Funding

None.

Recognizing the increasingly important role of point-of-care ultrasound (POCUS) in advancing clinical care, the Society of Hospital Medicine (SHM) has published a valuable series of position statements to guide hospitalists and administrators on the safe and effective use of POCUS.¹ In this issue of the Journal of Hospital Medicine, Soni et al. present a series of consensus-based recommendations on ultrasound guidance for lumbar puncture (LP).² Among these are the recommendations that ultrasound “should be used” to map the lumbar spine and to select an appropriate puncture site to reduce insertion attempts, reduce needle redirections, and increase overall procedural success.

At first glance, the recommendations appear definitive. However, not immediately obvious is the authors’ clarification that “This position statement does not mandate that hospitalists use ultrasound guidance for LP, nor does it establish ultrasound guidance as the standard of care for LP.” Even with the authors’ caveat, this nuance may not be readily apparent to the readers who review only the executive summary of the guidelines or who omit the context provided in the background of the position statement.

The directive language of this position statement may be a result of an unmerited amplification. The SHM POCUS Task Force employed the Research and Development Appropriateness Method to quantify the degree of consensus and the strength of the recommendation assigned,³ reaching “very good” consensus for each of the recommendations espoused in its position statement. Procedurally, this implies that ≥80% of the 27 voting members rated each published recommendation statement as “appropriate”. Using wording assigned a priori by the committee to each level of consensus, appropriateness became magnified to the declaration “should be used”. In this manner, the strength of the recommendations in this position statement is not necessarily based on the experts’ convictions related to ultrasound-guided LP, nor the strength of the supporting evidence.

In the case of ultrasound-guided LP, we might choose different descriptors than “appropriate” or “should be used”. The evidence base for ultrasound guidance for LP, though growing, may be insufficient as a foundation to a position statement and is certainly insufficient to create a new standard of care for hospitalists. Although the SHM POCUS Task Force completed a thoughtful literature review, no systematic approach (eg, GRADE methodology⁴) was used to rate the quality of evidence. Furthermore, the literature reviewed was drawn predominantly from anesthesia and emergency medicine sources—not readily generalizable to the hospitalist. Notably, these studies examined all neuraxial procedures (most commonly epidural and spinal anesthesia), which employ different techniques and tools than LP and are performed by clinicians with vastly different procedural training backgrounds than most hospitalists. Altogether, this creates the potential for a gap between true evidence quality and the strength of recommendation.

At a high level, although the technique for ultrasound mapping of the lumbar spine may be similar, the use of ultrasound has been less well studied specifically for LP. When considering LP alone, the available literature is inadequate to recommend uniform ultrasound guidance. A 2018 meta-analysis by Gottlieb et al. included 12 studies focusing only on LP, totaling N = 957 patients.⁵ This showed some favorability of ultrasound guidance, with a success rate of 90% using ultrasound, 81.4% with a landmark-based approach, and an odds ratio of 2.22 favoring ultrasound guidance (95% CI: 1.03-4.77). Unfortunately, when focusing only on adult patients, the advantage of POCUS diminished, with 91.4% success in the ultrasound group, 87.7% success in the landmark group, and a nonsignificant odds ratio of 2.10 (95% CI: 0.66-7.44).

Unequivocally, POCUS has established itself as a transformative technology for the guidance of invasive bedside procedures, bringing increased procedural success, improved safety, and decreased complication rates.⁶ For some procedures, particularly central venous catheterization, ultrasound guidance is a clear standard of care.^7,8 For LP, the greatest benefit has been observed in patients with anticipated procedural challenges, most commonly obese patients in whom landmarks are not easily palpable.⁹ Moreover, the harms ultrasound seeks to prevent are substantially different. The primary risk of deferring ultrasound guidance for LP is most often a failed procedure, whereas for other common ultrasound-guided procedures, the harms may include significant vascular injury, pneumothorax, or bowel perforation. Differences in the relative harms make risk-benefit assessments harder to quantify and studies harder to carry out.

Sonographic guidance for LP has a role in clinical practice and should always be considered. However, at present, there exist no guidelines in any other specialty regarding the routine use of ultrasound-guided LP, including anesthesia, emergency medicine, neurology, or interventional radiology.^10-15 As a result, a conservative interpretation of the POCUS Task Force’s findings would be to consider the use of ultrasound guidance for LP in patients where landmark identification is particularly challenging, but not to consider it a standard requirement for accreditation, training, or practice as of yet. Saying “more studies are required” can be a cop-out in some cases, but in this situation, the old adage does seem to apply.

We have great respect for the work of the SHM POCUS Task Force in advancing the use of POCUS in hospital medicine. Though ultrasound is not currently mandated as a care standard for the performance of LP, we all can agree that POCUS does confer advantages for this procedure, particularly in a well-selected patient population. To continue to provide care of the highest quality, hospitalists must be encouraged to elevate their practice with POCUS and be supported with the equipment, training, credentialing, and quality assurance structures necessary to integrate bedside ultrasound safely and effectively into their diagnostic and procedural practice.

Disclosures

No conflicts of interest to disclose.

Funding

None.

Respiratory rate (RR) is an essential vital sign that is routinely measured for hospitalized adults. It is a strong predictor of adverse events.^1,2 Therefore, RR is a key component of several widely used risk prediction scores, including the systemic inflammatory response syndrome (SIRS).³

Despite its clinical utility, RR is inaccurately measured.^4-7 One reason for the inaccurate measurement of RR is that RR measurement, in contrast to that of other vital signs, is not automated. The gold-standard technique for measuring RR is the visual assessment of a resting patient. Thus, RR measurement is perceived as time-consuming. Clinical staff instead frequently approximate RR through brief observation.^8-11

Given its clinical importance and widespread inaccuracy, we conducted a quality improvement (QI) initiative to improve RR accuracy.

Design and Setting

We conducted an interdisciplinary QI initiative by using the plan–do–study–act (PDSA) methodology from July 2017 to February 2018. The initiative was set in a single adult 28-bed medical inpatient unit of a large, urban, safety-net hospital consisting of general internal medicine and hematology/oncology patients. Routine vital sign measurements on this unit occur at four- or six-hour intervals per physician orders and are performed by patient-care assistants (PCAs) who are nonregistered nursing support staff. PCAs use a vital signs cart equipped with automated tools to measure vital signs except for RR, which is manually assessed. PCAs are trained on vital sign measurements during a two-day onboarding orientation and four to six weeks of on-the-job training by experienced PCAs. PCAs are directly supervised by nursing operations managers. Formal continuing education programs for PCAs or performance audits of their clinical duties did not exist prior to our QI initiative.

Intervention

Intervention development addressing several important barriers and workflow inefficiencies was based on the direct observation of PCA workflow and information gathering by engaging stakeholders, including PCAs, nursing operations management, nursing leadership, and hospital administration (PDSA cycles 1-7 in Table). Our modified PCA vital sign workflow incorporated RR measurement during the approximate 30 seconds needed to complete automated blood pressure measurement as previously described.¹² Nursing administration purchased three stopwatches (each $5 US) to attach to vital signs carts. One investigator (NK) participated in two monthly one-hour meetings, and three investigators (NK, KB, and SD) participated in 19 daily 15-minute huddles to conduct stakeholder engagement and educate and retrain PCAs on proper technique (total of 6.75 hours).

Evaluation

The primary aim of this QI initiative was to improve RR accuracy, which was evaluated using two distinct but complementary analyses: the prospective comparison of PCA-recorded RRs with gold-standard recorded RRs and the retrospective comparison of RRs recorded in electronic health records (EHR) on the intervention unit versus two control units. The secondary aims were to examine time to complete vital sign measurement and to assess whether the intervention was associated with a reduction in the incidence of SIRS specifically due to tachypnea.

Respiratory Rate Accuracy

PCA-recorded RRs were considered accurate if the RR was within ±2 breaths of a gold-standard RR measurement performed by a trained study member (NK or KB). We conducted gold-standard RR measurements for 100 observations pre- and postintervention within 30 minutes of PCA measurement to avoid Hawthorne bias.

We assessed the variability of recorded RRs in the EHR for all patients in the intervention unit as a proxy for accuracy. We hypothesized on the basis of prior research that improving the accuracy of RR measurement would increase the variability and normality of distribution in RRs.¹³ This is an approach that we have employed previously.⁷ The EHR cohort included consecutive hospitalizations by patients who were admitted to either the intervention unit or to one of two nonintervention general medicine inpatient units that served as concurrent controls. We grouped hospitalizations into a preintervention phase from March 1, 2017-July 22, 2017, a planning phase from July 23, 2017-December 3, 2017, and a postintervention phase from December 21, 2017-February 28, 2018. Hospitalizations during the two-week teaching phase from December 3, 2017-December 21, 2017 were excluded. We excluded vital signs obtained in the emergency department or in a location different from the patient’s admission unit. We qualitatively assessed RR distribution using histograms as we have done previously.⁷

We examined the distributions of RRs recorded in the EHR before and after intervention by individual PCAs on the intervention floor to assess for fidelity and adherence in the PCA uptake of the intervention.

Time

We compared the time to complete vital sign measurement among convenience samples of 50 unique observations pre- and postintervention using the Wilcoxon rank sum test.

SIRS Incidence

Since we hypothesized that improved RR accuracy would reduce falsely elevated RRs but have no impact on the other three SIRS criteria, we assessed changes in tachypnea-specific SIRS incidence, which was defined a priori as the presence of exactly two concurrent SIRS criteria, one of which was an elevated RR.³ We examined changes using a difference-in-differences approach with three different units of analysis (per vital sign measurement, hospital-day, and hospitalization; see footnote for Appendix Table 1 for methodological details. All analyses were conducted using STATA 12.0 (StataCorp, College Station, Texas).

Respiratory Rate Accuracy

Prior to the intervention, the median PCA RR was 18 (IQR 18-20) versus 12 (IQR 12-18) for the gold-standard RR (Appendix Figure 1), with only 36% of PCA measurements considered accurate. After the intervention, the median PCA-recorded RR was 14 (IQR 15-20) versus 14 (IQR 14-20) for the gold-standard RR and a RR accuracy of 58% (P < .001).

For our analyses on RR distribution using EHR data, we included 143,447 unique RRs (Appendix Table 2). After the intervention, the normality of the distribution of RRs on the intervention unit had increased, whereas those of RRs on the control units remained qualitatively similar pre- and postintervention (Appendix Figure 2).

Notable differences existed among the 11 individual PCAs (Figure) despite observing increased variability in PCA-recorded RRs postintervention. Some PCAs (numbers 2, 7, and 10) shifted their narrow RR interquartile range lower by several breaths/minute, whereas most other PCAs had a reduced median RR and widened interquartile range.

Time

Before the intervention, the median time to complete vital sign measurements was 2:36 (IQR 2:04-3:20). After the intervention, the time to complete vital signs decreased to 1:55 (IQR, 1:40-2:22; P < .001), which was 41 less seconds on average per vital sign set.

SIRS Incidence

The intervention was associated with a 3.3% reduction (95% CI, –6.4% to –0.005%) in tachypnea-specific SIRS incidence per hospital-day and a 7.8% reduction (95% CI, –13.5% to –2.2%) per hospitalization (Appendix Table 1). We also observed a modest reduction in overall SIRS incidence after the intervention (2.9% less per vital sign check, 4.6% less per hospital-day, and 3.2% less per hospitalization), although these reductions were not statistically significant.

Our QI initiative improved the absolute RR accuracy by 22%, saved PCAs 41 seconds on average per vital sign measurement, and decreased the absolute proportion of hospitalizations with tachypnea-specific SIRS by 7.8%. Our intervention is a novel, interdisciplinary, low-cost, low-effort, low-tech approach that addressed known challenges to accurate RR measurement,^8,9,11 as well as the key barriers identified in our initial PDSA cycles. Our approach includes adding a time-keeping device to vital sign carts and standardizing a PCA vital sign workflow with increased efficiency. Lastly, this intervention is potentially scalable because stakeholder engagement, education, and retraining of the entire PCA staff for the unit required only 6.75 hours.

While our primary goal was to improve RR accuracy, our QI initiative also improved vital sign efficiency. By extrapolating our findings to an eight-hour PCA shift caring for eight patients who require vital sign checks every four hours, we estimated that our intervention would save approximately 16:24 minutes per PCA shift. This newfound time could be repurposed for other patient-care tasks or could be spent ensuring the accuracy of other vital signs given that accurate monitoring may be neglected because of time constraints.¹¹ Additionally, the improvement in RR accuracy reduced falsely elevated RRs and thus lowered SIRS incidence specifically due to tachypnea. Given that EHR-based sepsis alerts are often based on SIRS criteria, improved RR accuracy may also improve alarm fatigue by reducing the rate of false-positive alerts.¹⁴

This initiative is not without limitations. Generalizability to other hospitals and even other units within the same hospital is uncertain. However, because this initiative was conducted within a safety-net hospital, we anticipate at least similar, if not increased, success in better-resourced hospitals. Second, the long-term durability of our intervention is unclear, although EHR RR variability remained steady for two months after our intervention (data not shown).

To ensure long-term sustainability and further improve RR accuracy, future PDSA cycles could include electing a PCA “vital signs champion” to reiterate the importance of RRs in clinical decision-making and ensure adherence to the modified workflow. Nursing champions act as persuasive change agents that disseminate and implement healthcare change,¹⁵ which may also be true of PCA champions. Additionally, future PDSA cycles can obviate the need for labor-intensive manual audits by leveraging EHR-based auditing to target education and retraining interventions to PCAs with minimal RR variability to optimize workflow adherence.

In conclusion, through a multipronged QI initiative we improved RR accuracy, increased the efficiency of vital sign measurement, and decreased SIRS incidence specifically due to tachypnea by reducing the number of falsely elevated RRs. This novel, low-cost, low-effort, low-tech approach can readily be implemented and disseminated in hospital inpatient settings.

Acknowledgments

The authors would like to acknowledge the meaningful contributions of Mr. Sudarshaan Pathak, RN, Ms. Shirly Koduvathu, RN, and Ms. Judy Herrington MSN, RN in this multidisciplinary initiative. We thank Mr. Christopher McKintosh, RN for his support in data acquisition. Lastly, the authors would like to acknowledge all of the patient-care assistants involved in this QI initiative.

Disclosures

Dr. Makam reports grants from NIA/NIH, during the conduct of the study. All other authors have nothing to disclose.

Funding

This work is supported in part by the Agency for Healthcare Research and Quality-funded UT Southwestern Center for Patient-Centered Outcomes Research (R24HS022418). OKN is funded by the National Heart, Lung, and Blood Institute (K23HL133441), and ANM is funded by the National Institute on Aging (K23AG052603).

Respiratory rate (RR) is an essential vital sign that is routinely measured for hospitalized adults. It is a strong predictor of adverse events.^1,2 Therefore, RR is a key component of several widely used risk prediction scores, including the systemic inflammatory response syndrome (SIRS).³

Despite its clinical utility, RR is inaccurately measured.^4-7 One reason for the inaccurate measurement of RR is that RR measurement, in contrast to that of other vital signs, is not automated. The gold-standard technique for measuring RR is the visual assessment of a resting patient. Thus, RR measurement is perceived as time-consuming. Clinical staff instead frequently approximate RR through brief observation.^8-11

Given its clinical importance and widespread inaccuracy, we conducted a quality improvement (QI) initiative to improve RR accuracy.

Design and Setting

We conducted an interdisciplinary QI initiative by using the plan–do–study–act (PDSA) methodology from July 2017 to February 2018. The initiative was set in a single adult 28-bed medical inpatient unit of a large, urban, safety-net hospital consisting of general internal medicine and hematology/oncology patients. Routine vital sign measurements on this unit occur at four- or six-hour intervals per physician orders and are performed by patient-care assistants (PCAs) who are nonregistered nursing support staff. PCAs use a vital signs cart equipped with automated tools to measure vital signs except for RR, which is manually assessed. PCAs are trained on vital sign measurements during a two-day onboarding orientation and four to six weeks of on-the-job training by experienced PCAs. PCAs are directly supervised by nursing operations managers. Formal continuing education programs for PCAs or performance audits of their clinical duties did not exist prior to our QI initiative.

Intervention

Intervention development addressing several important barriers and workflow inefficiencies was based on the direct observation of PCA workflow and information gathering by engaging stakeholders, including PCAs, nursing operations management, nursing leadership, and hospital administration (PDSA cycles 1-7 in Table). Our modified PCA vital sign workflow incorporated RR measurement during the approximate 30 seconds needed to complete automated blood pressure measurement as previously described.¹² Nursing administration purchased three stopwatches (each $5 US) to attach to vital signs carts. One investigator (NK) participated in two monthly one-hour meetings, and three investigators (NK, KB, and SD) participated in 19 daily 15-minute huddles to conduct stakeholder engagement and educate and retrain PCAs on proper technique (total of 6.75 hours).

Evaluation

The primary aim of this QI initiative was to improve RR accuracy, which was evaluated using two distinct but complementary analyses: the prospective comparison of PCA-recorded RRs with gold-standard recorded RRs and the retrospective comparison of RRs recorded in electronic health records (EHR) on the intervention unit versus two control units. The secondary aims were to examine time to complete vital sign measurement and to assess whether the intervention was associated with a reduction in the incidence of SIRS specifically due to tachypnea.

Respiratory Rate Accuracy

PCA-recorded RRs were considered accurate if the RR was within ±2 breaths of a gold-standard RR measurement performed by a trained study member (NK or KB). We conducted gold-standard RR measurements for 100 observations pre- and postintervention within 30 minutes of PCA measurement to avoid Hawthorne bias.

We assessed the variability of recorded RRs in the EHR for all patients in the intervention unit as a proxy for accuracy. We hypothesized on the basis of prior research that improving the accuracy of RR measurement would increase the variability and normality of distribution in RRs.¹³ This is an approach that we have employed previously.⁷ The EHR cohort included consecutive hospitalizations by patients who were admitted to either the intervention unit or to one of two nonintervention general medicine inpatient units that served as concurrent controls. We grouped hospitalizations into a preintervention phase from March 1, 2017-July 22, 2017, a planning phase from July 23, 2017-December 3, 2017, and a postintervention phase from December 21, 2017-February 28, 2018. Hospitalizations during the two-week teaching phase from December 3, 2017-December 21, 2017 were excluded. We excluded vital signs obtained in the emergency department or in a location different from the patient’s admission unit. We qualitatively assessed RR distribution using histograms as we have done previously.⁷

We examined the distributions of RRs recorded in the EHR before and after intervention by individual PCAs on the intervention floor to assess for fidelity and adherence in the PCA uptake of the intervention.

Time

We compared the time to complete vital sign measurement among convenience samples of 50 unique observations pre- and postintervention using the Wilcoxon rank sum test.

SIRS Incidence

Since we hypothesized that improved RR accuracy would reduce falsely elevated RRs but have no impact on the other three SIRS criteria, we assessed changes in tachypnea-specific SIRS incidence, which was defined a priori as the presence of exactly two concurrent SIRS criteria, one of which was an elevated RR.³ We examined changes using a difference-in-differences approach with three different units of analysis (per vital sign measurement, hospital-day, and hospitalization; see footnote for Appendix Table 1 for methodological details. All analyses were conducted using STATA 12.0 (StataCorp, College Station, Texas).

Respiratory Rate Accuracy

Prior to the intervention, the median PCA RR was 18 (IQR 18-20) versus 12 (IQR 12-18) for the gold-standard RR (Appendix Figure 1), with only 36% of PCA measurements considered accurate. After the intervention, the median PCA-recorded RR was 14 (IQR 15-20) versus 14 (IQR 14-20) for the gold-standard RR and a RR accuracy of 58% (P < .001).

For our analyses on RR distribution using EHR data, we included 143,447 unique RRs (Appendix Table 2). After the intervention, the normality of the distribution of RRs on the intervention unit had increased, whereas those of RRs on the control units remained qualitatively similar pre- and postintervention (Appendix Figure 2).

Notable differences existed among the 11 individual PCAs (Figure) despite observing increased variability in PCA-recorded RRs postintervention. Some PCAs (numbers 2, 7, and 10) shifted their narrow RR interquartile range lower by several breaths/minute, whereas most other PCAs had a reduced median RR and widened interquartile range.

Time

Before the intervention, the median time to complete vital sign measurements was 2:36 (IQR 2:04-3:20). After the intervention, the time to complete vital signs decreased to 1:55 (IQR, 1:40-2:22; P < .001), which was 41 less seconds on average per vital sign set.

SIRS Incidence

The intervention was associated with a 3.3% reduction (95% CI, –6.4% to –0.005%) in tachypnea-specific SIRS incidence per hospital-day and a 7.8% reduction (95% CI, –13.5% to –2.2%) per hospitalization (Appendix Table 1). We also observed a modest reduction in overall SIRS incidence after the intervention (2.9% less per vital sign check, 4.6% less per hospital-day, and 3.2% less per hospitalization), although these reductions were not statistically significant.

Our QI initiative improved the absolute RR accuracy by 22%, saved PCAs 41 seconds on average per vital sign measurement, and decreased the absolute proportion of hospitalizations with tachypnea-specific SIRS by 7.8%. Our intervention is a novel, interdisciplinary, low-cost, low-effort, low-tech approach that addressed known challenges to accurate RR measurement,^8,9,11 as well as the key barriers identified in our initial PDSA cycles. Our approach includes adding a time-keeping device to vital sign carts and standardizing a PCA vital sign workflow with increased efficiency. Lastly, this intervention is potentially scalable because stakeholder engagement, education, and retraining of the entire PCA staff for the unit required only 6.75 hours.

While our primary goal was to improve RR accuracy, our QI initiative also improved vital sign efficiency. By extrapolating our findings to an eight-hour PCA shift caring for eight patients who require vital sign checks every four hours, we estimated that our intervention would save approximately 16:24 minutes per PCA shift. This newfound time could be repurposed for other patient-care tasks or could be spent ensuring the accuracy of other vital signs given that accurate monitoring may be neglected because of time constraints.¹¹ Additionally, the improvement in RR accuracy reduced falsely elevated RRs and thus lowered SIRS incidence specifically due to tachypnea. Given that EHR-based sepsis alerts are often based on SIRS criteria, improved RR accuracy may also improve alarm fatigue by reducing the rate of false-positive alerts.¹⁴

This initiative is not without limitations. Generalizability to other hospitals and even other units within the same hospital is uncertain. However, because this initiative was conducted within a safety-net hospital, we anticipate at least similar, if not increased, success in better-resourced hospitals. Second, the long-term durability of our intervention is unclear, although EHR RR variability remained steady for two months after our intervention (data not shown).

To ensure long-term sustainability and further improve RR accuracy, future PDSA cycles could include electing a PCA “vital signs champion” to reiterate the importance of RRs in clinical decision-making and ensure adherence to the modified workflow. Nursing champions act as persuasive change agents that disseminate and implement healthcare change,¹⁵ which may also be true of PCA champions. Additionally, future PDSA cycles can obviate the need for labor-intensive manual audits by leveraging EHR-based auditing to target education and retraining interventions to PCAs with minimal RR variability to optimize workflow adherence.

In conclusion, through a multipronged QI initiative we improved RR accuracy, increased the efficiency of vital sign measurement, and decreased SIRS incidence specifically due to tachypnea by reducing the number of falsely elevated RRs. This novel, low-cost, low-effort, low-tech approach can readily be implemented and disseminated in hospital inpatient settings.

Acknowledgments

The authors would like to acknowledge the meaningful contributions of Mr. Sudarshaan Pathak, RN, Ms. Shirly Koduvathu, RN, and Ms. Judy Herrington MSN, RN in this multidisciplinary initiative. We thank Mr. Christopher McKintosh, RN for his support in data acquisition. Lastly, the authors would like to acknowledge all of the patient-care assistants involved in this QI initiative.

Disclosures

Dr. Makam reports grants from NIA/NIH, during the conduct of the study. All other authors have nothing to disclose.

Funding

This work is supported in part by the Agency for Healthcare Research and Quality-funded UT Southwestern Center for Patient-Centered Outcomes Research (R24HS022418). OKN is funded by the National Heart, Lung, and Blood Institute (K23HL133441), and ANM is funded by the National Institute on Aging (K23AG052603).

Respiratory rate (RR) is an essential vital sign that is routinely measured for hospitalized adults. It is a strong predictor of adverse events.^1,2 Therefore, RR is a key component of several widely used risk prediction scores, including the systemic inflammatory response syndrome (SIRS).³

Despite its clinical utility, RR is inaccurately measured.^4-7 One reason for the inaccurate measurement of RR is that RR measurement, in contrast to that of other vital signs, is not automated. The gold-standard technique for measuring RR is the visual assessment of a resting patient. Thus, RR measurement is perceived as time-consuming. Clinical staff instead frequently approximate RR through brief observation.^8-11

Given its clinical importance and widespread inaccuracy, we conducted a quality improvement (QI) initiative to improve RR accuracy.

Design and Setting

We conducted an interdisciplinary QI initiative by using the plan–do–study–act (PDSA) methodology from July 2017 to February 2018. The initiative was set in a single adult 28-bed medical inpatient unit of a large, urban, safety-net hospital consisting of general internal medicine and hematology/oncology patients. Routine vital sign measurements on this unit occur at four- or six-hour intervals per physician orders and are performed by patient-care assistants (PCAs) who are nonregistered nursing support staff. PCAs use a vital signs cart equipped with automated tools to measure vital signs except for RR, which is manually assessed. PCAs are trained on vital sign measurements during a two-day onboarding orientation and four to six weeks of on-the-job training by experienced PCAs. PCAs are directly supervised by nursing operations managers. Formal continuing education programs for PCAs or performance audits of their clinical duties did not exist prior to our QI initiative.

Intervention

Intervention development addressing several important barriers and workflow inefficiencies was based on the direct observation of PCA workflow and information gathering by engaging stakeholders, including PCAs, nursing operations management, nursing leadership, and hospital administration (PDSA cycles 1-7 in Table). Our modified PCA vital sign workflow incorporated RR measurement during the approximate 30 seconds needed to complete automated blood pressure measurement as previously described.¹² Nursing administration purchased three stopwatches (each $5 US) to attach to vital signs carts. One investigator (NK) participated in two monthly one-hour meetings, and three investigators (NK, KB, and SD) participated in 19 daily 15-minute huddles to conduct stakeholder engagement and educate and retrain PCAs on proper technique (total of 6.75 hours).

Evaluation

The primary aim of this QI initiative was to improve RR accuracy, which was evaluated using two distinct but complementary analyses: the prospective comparison of PCA-recorded RRs with gold-standard recorded RRs and the retrospective comparison of RRs recorded in electronic health records (EHR) on the intervention unit versus two control units. The secondary aims were to examine time to complete vital sign measurement and to assess whether the intervention was associated with a reduction in the incidence of SIRS specifically due to tachypnea.

Respiratory Rate Accuracy

PCA-recorded RRs were considered accurate if the RR was within ±2 breaths of a gold-standard RR measurement performed by a trained study member (NK or KB). We conducted gold-standard RR measurements for 100 observations pre- and postintervention within 30 minutes of PCA measurement to avoid Hawthorne bias.

We assessed the variability of recorded RRs in the EHR for all patients in the intervention unit as a proxy for accuracy. We hypothesized on the basis of prior research that improving the accuracy of RR measurement would increase the variability and normality of distribution in RRs.¹³ This is an approach that we have employed previously.⁷ The EHR cohort included consecutive hospitalizations by patients who were admitted to either the intervention unit or to one of two nonintervention general medicine inpatient units that served as concurrent controls. We grouped hospitalizations into a preintervention phase from March 1, 2017-July 22, 2017, a planning phase from July 23, 2017-December 3, 2017, and a postintervention phase from December 21, 2017-February 28, 2018. Hospitalizations during the two-week teaching phase from December 3, 2017-December 21, 2017 were excluded. We excluded vital signs obtained in the emergency department or in a location different from the patient’s admission unit. We qualitatively assessed RR distribution using histograms as we have done previously.⁷

We examined the distributions of RRs recorded in the EHR before and after intervention by individual PCAs on the intervention floor to assess for fidelity and adherence in the PCA uptake of the intervention.

Time

We compared the time to complete vital sign measurement among convenience samples of 50 unique observations pre- and postintervention using the Wilcoxon rank sum test.

SIRS Incidence

Since we hypothesized that improved RR accuracy would reduce falsely elevated RRs but have no impact on the other three SIRS criteria, we assessed changes in tachypnea-specific SIRS incidence, which was defined a priori as the presence of exactly two concurrent SIRS criteria, one of which was an elevated RR.³ We examined changes using a difference-in-differences approach with three different units of analysis (per vital sign measurement, hospital-day, and hospitalization; see footnote for Appendix Table 1 for methodological details. All analyses were conducted using STATA 12.0 (StataCorp, College Station, Texas).

Respiratory Rate Accuracy

Prior to the intervention, the median PCA RR was 18 (IQR 18-20) versus 12 (IQR 12-18) for the gold-standard RR (Appendix Figure 1), with only 36% of PCA measurements considered accurate. After the intervention, the median PCA-recorded RR was 14 (IQR 15-20) versus 14 (IQR 14-20) for the gold-standard RR and a RR accuracy of 58% (P < .001).

For our analyses on RR distribution using EHR data, we included 143,447 unique RRs (Appendix Table 2). After the intervention, the normality of the distribution of RRs on the intervention unit had increased, whereas those of RRs on the control units remained qualitatively similar pre- and postintervention (Appendix Figure 2).

Notable differences existed among the 11 individual PCAs (Figure) despite observing increased variability in PCA-recorded RRs postintervention. Some PCAs (numbers 2, 7, and 10) shifted their narrow RR interquartile range lower by several breaths/minute, whereas most other PCAs had a reduced median RR and widened interquartile range.

Time

Before the intervention, the median time to complete vital sign measurements was 2:36 (IQR 2:04-3:20). After the intervention, the time to complete vital signs decreased to 1:55 (IQR, 1:40-2:22; P < .001), which was 41 less seconds on average per vital sign set.

SIRS Incidence

The intervention was associated with a 3.3% reduction (95% CI, –6.4% to –0.005%) in tachypnea-specific SIRS incidence per hospital-day and a 7.8% reduction (95% CI, –13.5% to –2.2%) per hospitalization (Appendix Table 1). We also observed a modest reduction in overall SIRS incidence after the intervention (2.9% less per vital sign check, 4.6% less per hospital-day, and 3.2% less per hospitalization), although these reductions were not statistically significant.

Our QI initiative improved the absolute RR accuracy by 22%, saved PCAs 41 seconds on average per vital sign measurement, and decreased the absolute proportion of hospitalizations with tachypnea-specific SIRS by 7.8%. Our intervention is a novel, interdisciplinary, low-cost, low-effort, low-tech approach that addressed known challenges to accurate RR measurement,^8,9,11 as well as the key barriers identified in our initial PDSA cycles. Our approach includes adding a time-keeping device to vital sign carts and standardizing a PCA vital sign workflow with increased efficiency. Lastly, this intervention is potentially scalable because stakeholder engagement, education, and retraining of the entire PCA staff for the unit required only 6.75 hours.

While our primary goal was to improve RR accuracy, our QI initiative also improved vital sign efficiency. By extrapolating our findings to an eight-hour PCA shift caring for eight patients who require vital sign checks every four hours, we estimated that our intervention would save approximately 16:24 minutes per PCA shift. This newfound time could be repurposed for other patient-care tasks or could be spent ensuring the accuracy of other vital signs given that accurate monitoring may be neglected because of time constraints.¹¹ Additionally, the improvement in RR accuracy reduced falsely elevated RRs and thus lowered SIRS incidence specifically due to tachypnea. Given that EHR-based sepsis alerts are often based on SIRS criteria, improved RR accuracy may also improve alarm fatigue by reducing the rate of false-positive alerts.¹⁴

This initiative is not without limitations. Generalizability to other hospitals and even other units within the same hospital is uncertain. However, because this initiative was conducted within a safety-net hospital, we anticipate at least similar, if not increased, success in better-resourced hospitals. Second, the long-term durability of our intervention is unclear, although EHR RR variability remained steady for two months after our intervention (data not shown).

To ensure long-term sustainability and further improve RR accuracy, future PDSA cycles could include electing a PCA “vital signs champion” to reiterate the importance of RRs in clinical decision-making and ensure adherence to the modified workflow. Nursing champions act as persuasive change agents that disseminate and implement healthcare change,¹⁵ which may also be true of PCA champions. Additionally, future PDSA cycles can obviate the need for labor-intensive manual audits by leveraging EHR-based auditing to target education and retraining interventions to PCAs with minimal RR variability to optimize workflow adherence.

In conclusion, through a multipronged QI initiative we improved RR accuracy, increased the efficiency of vital sign measurement, and decreased SIRS incidence specifically due to tachypnea by reducing the number of falsely elevated RRs. This novel, low-cost, low-effort, low-tech approach can readily be implemented and disseminated in hospital inpatient settings.

Acknowledgments

The authors would like to acknowledge the meaningful contributions of Mr. Sudarshaan Pathak, RN, Ms. Shirly Koduvathu, RN, and Ms. Judy Herrington MSN, RN in this multidisciplinary initiative. We thank Mr. Christopher McKintosh, RN for his support in data acquisition. Lastly, the authors would like to acknowledge all of the patient-care assistants involved in this QI initiative.

Disclosures

Dr. Makam reports grants from NIA/NIH, during the conduct of the study. All other authors have nothing to disclose.

Funding

This work is supported in part by the Agency for Healthcare Research and Quality-funded UT Southwestern Center for Patient-Centered Outcomes Research (R24HS022418). OKN is funded by the National Heart, Lung, and Blood Institute (K23HL133441), and ANM is funded by the National Institute on Aging (K23AG052603).

Evidence exists that physicians consider what may be called “social” or “nonmedical” factors (lack of social support or barriers to access) in hospital admission decision-making and that patients are hospitalized even in the absence of a level of medical acuity warranting admission.^1-3 Although hospitalization is associated with the risk of adverse events (AEs),⁴ whether this risk is related to the medical acuity of admission remains unclear. Our study sought to quantify the AEs experienced by patients hospitalized without definite medical acuity compared with those experienced by patients hospitalized with a definite medically appropriate indication for admission.

Setting and Database Used for Analysis

This study was conducted at an urban, safety-net, public teaching hospital. At our site, calls for medical admissions are always answered by a hospital medicine attending physician (“triage physician”) who works collaboratively with the referring physician to facilitate appropriate disposition. Many of these discussions occur via telephone, but the triage physician may also assess the patient directly if needed. This study involved 24 triage physicians who directly assessed the patient in 65% of the cases.

At the time of each admission call, the triage physician logs the following information into a central triage database: date and time of call, patient location, reason for admission, assessment of appropriateness for medical floor, contributing factors to admission decision-making, and patient disposition.

Admission Appropriateness Group Designation

To be considered for inclusion in this study, calls must have originated from the emergency department and resulted in admission to the general medicine floor on either a resident teaching or hospitalist service from February 1, 2018 to June 1, 2018. This time frame was selected to avoid the start of a new academic cycle in late June that may confound AE rates.

The designation of appropriateness was determined by the triage physician’s logged response to triage database questions at the time of the admission call. Of the 748 admissions meeting inclusion criteria, 513 (68.6%) were considered definitely appropriate on the basis of the triage physician’s response to the question “Based ONLY on the medical reason for hospitalization, in your opinion, how appropriate is this admission to the medicine floor service?” Furthermore, 169 (22.6%) were considered without definite medical acuity on the basis of the triage physician’s indication that “severity of medical problems alone may not require inpatient hospitalization” (Appendix Figure 1).

Study Design

Following a retrospective cohort study design, we systematically sampled 150 admissions from those “admitted without definite medical acuity” to create the exposure group and 150 from the “definitely medically appropriate” admissions to create the nonexposure group. Our sampling method involved selecting every third record until reaching the target sample size. This method and group sizes were determined prior to beginning data collection. Given the expected incidence of 33% AEs in the unexposed group (consistent with previous reports of AEs using the trigger tool⁵), we anticipated that a total sample size of 300 would be appropriate to capture a relative risk of at least 1.5 with 80% power and 95% confidence level.

Chart review was performed to capture patient demographics, admission characteristics, and hospitalization outcomes. We captured emergency severity index (ESI)⁶, a validated, reliable triage assessment score assigned by our emergency department, as a measurement of acute illness and calculated the Charlson comorbidity index (CCI)⁷ as a measurement of chronic comorbidity.

Identification of Adverse Events

We measured AEs by using the Institute for Healthcare Improvement Global Trigger Tool,^8,9 which is estimated to identify up to 10 times more AEs than other methods, such as voluntary reporting.⁵ This protocol includes 28 triggers in the Cares and Medication Modules that serve as indicators that an AE may have occurred. The presence of a trigger is not necessarily an AE but a clue for further analysis. Two investigators (AS and CS) independently systematically searched for the presence of triggers within each patient chart. Trigger identification prompted in-depth analysis to confirm the occurrence of an AE and to characterize its severity by using the National Coordinating Council for Medication Error Reporting and Prevention categorization.¹⁰ An AE was coded when independent reviewers identified evidence of a preventable or nonpreventable “noxious and unintended event occurring in association with medical care.”⁹ By definition, any AEs identified were patient harms. Findings were reviewed weekly to ensure agreement, and discrepancies were adjudicated by a third investigator (MB).

All study data were collected by using REDCap electronic data capture tools hosted at the University of Washington.¹¹ The University of Washington Institutional Review Board granted approval for this study.

Study Outcome and Statistical Analysis

The primary outcome was AEs per group with results calculated in three ways: AEs per 1,000 patient-days, AEs per 100 admissions, and percent of admissions with an AE. The risk ratio (RR) for the percent of admissions with an AE and the incidence rate ratio (IRR) for AEs per 1,000 patient-days were calculated for the comparison of significance.

Other data were analyzed by using Pearson’s chi square for categorical data, Student t test for normally distributed quantitative data, and Wilcoxon rank-sum (Mann–Whitney) for the length of stay (due to skew). Analyses were conducted using STATA (version 15.1, College Station, TX).

This work follows standards for reporting observational students as outlined in the STROBE statement.¹²

Patient Demographics

Both groups were predominantly white/non-Hispanic, male, and English-speaking (Table 1). More patients without definite medical acuity were covered by public insurance (78.9% vs 69.8%, P = .010) and discharged to homelessness (34.8% vs 22.6%, P = .041).

Measures of Illness

Patients considered definitely medically appropriate had lower ESI scores, indicative of more acute presentation, than those without definite medical acuity (2.73 [95% CI 2.64-2.81] vs 2.87 [95% CI 2.78-2.95], P = .026). There was no difference in CCI scores (Table 1).

Reason for Admission and Outcomes

Admissions considered definitely medically appropriate more frequently had an identified diagnosis/syndrome (66% vs 53%) or objective measurement (8.7% vs 2.7%) listed as the reason for admission, whereas patients admitted without definite medical acuity more freuqently had undifferentiated symptoms (34.7% vs 24%) or other/disposition (6% vs 1.3%) listed. The most common factors that triage physicians cited as contributing to the decision to admit patients without definite medical acuity included homelessness (34%), lack of outpatient social support (32%), and substance use disorder (25%). More details are available in Appendix Tables 1 and 2.

Admissions without definite medical acuity were longer than those with definite medical acuity (6.6 vs 6.0 days, P = .038), but there was no difference in emergency department readmissions within 48 hours or hospital readmissions within 30 days (Table 1).

Adverse Events

We identified 76 AEs in 41 admissions without definite medical acuity (range 0-10 AEs per admission) and 63 AEs in 44 definitely medically appropriate admissions (range 0-4 AEs per admission). The percentage of admissions with AE (27.3% vs 29.3%; RR 0.93, 95% CI 0.65-1.34, P = .70) and the rate of AE/1,000 patient-days (76.8 vs 70.4; IRR 1.09, 95% CI 0.77-1.55, P = .61) did not show statistically significant differences. The distribution of AE severity was similar between the two groups (Table 2). Most identified AEs caused temporary harm to the patient and were rated at severity levels E or F. Severe AEs, including at least one level I (patient death), occurred in both groups. The complete listing of positive triggers leading to adverse event identification by group and severity is available in Appendix Table 3.

By using a robust, standardized method, we found that patients admitted without definite medical acuity experienced the same number of inpatient AEs as patients admitted for definitely medically appropriate reasons. While the groups were relatively similar overall in terms of demographics and chronic comorbidity, we found evidence of social vulnerability in the group admitted without definite medical acuity in the form of increased rates of homelessness, triage physician concern regarding the lack of outpatient social support, and disposition-related reasons for admission. That both groups suffered harm―including patient death―while admitted to the hospital is striking, in particular for those patients who were admitted because of the lack of suitable outpatient options.

The potential limitations to the generalizability of this work include the single-site, safety-net setting and the use of individual physician determination of admission appropriateness. The proportion of admissions without definite medical acuity reported here is similar to that reported by previously published admission decision-making studies,^2,3 and the rate of AEs observed is similar to rates measured in other studies using the trigger tool methodology.^5,13 These similarities suggest some commonality across settings. Our study treats triage physician assessment as the marker of difference in defining the two groups and is an inherently subjective assessment that is reflective of real-world, holistic decision-making. Notably, the triage physician assessment was corroborated by corresponding differences in the ESI score, an acute triage assessment completed by a clinician outside of our team.

This study adds foundational knowledge to the risk/benefit discussion surrounding the decision to admit. Physician admission decisions are likely influenced by concern for the safety of vulnerable patients. Our results suggest that considering the risk of hospitalization itself in this decision-making remains important.

Evidence exists that physicians consider what may be called “social” or “nonmedical” factors (lack of social support or barriers to access) in hospital admission decision-making and that patients are hospitalized even in the absence of a level of medical acuity warranting admission.^1-3 Although hospitalization is associated with the risk of adverse events (AEs),⁴ whether this risk is related to the medical acuity of admission remains unclear. Our study sought to quantify the AEs experienced by patients hospitalized without definite medical acuity compared with those experienced by patients hospitalized with a definite medically appropriate indication for admission.

Setting and Database Used for Analysis

This study was conducted at an urban, safety-net, public teaching hospital. At our site, calls for medical admissions are always answered by a hospital medicine attending physician (“triage physician”) who works collaboratively with the referring physician to facilitate appropriate disposition. Many of these discussions occur via telephone, but the triage physician may also assess the patient directly if needed. This study involved 24 triage physicians who directly assessed the patient in 65% of the cases.

At the time of each admission call, the triage physician logs the following information into a central triage database: date and time of call, patient location, reason for admission, assessment of appropriateness for medical floor, contributing factors to admission decision-making, and patient disposition.

Admission Appropriateness Group Designation

To be considered for inclusion in this study, calls must have originated from the emergency department and resulted in admission to the general medicine floor on either a resident teaching or hospitalist service from February 1, 2018 to June 1, 2018. This time frame was selected to avoid the start of a new academic cycle in late June that may confound AE rates.

The designation of appropriateness was determined by the triage physician’s logged response to triage database questions at the time of the admission call. Of the 748 admissions meeting inclusion criteria, 513 (68.6%) were considered definitely appropriate on the basis of the triage physician’s response to the question “Based ONLY on the medical reason for hospitalization, in your opinion, how appropriate is this admission to the medicine floor service?” Furthermore, 169 (22.6%) were considered without definite medical acuity on the basis of the triage physician’s indication that “severity of medical problems alone may not require inpatient hospitalization” (Appendix Figure 1).

Study Design

Following a retrospective cohort study design, we systematically sampled 150 admissions from those “admitted without definite medical acuity” to create the exposure group and 150 from the “definitely medically appropriate” admissions to create the nonexposure group. Our sampling method involved selecting every third record until reaching the target sample size. This method and group sizes were determined prior to beginning data collection. Given the expected incidence of 33% AEs in the unexposed group (consistent with previous reports of AEs using the trigger tool⁵), we anticipated that a total sample size of 300 would be appropriate to capture a relative risk of at least 1.5 with 80% power and 95% confidence level.

Chart review was performed to capture patient demographics, admission characteristics, and hospitalization outcomes. We captured emergency severity index (ESI)⁶, a validated, reliable triage assessment score assigned by our emergency department, as a measurement of acute illness and calculated the Charlson comorbidity index (CCI)⁷ as a measurement of chronic comorbidity.

Identification of Adverse Events

We measured AEs by using the Institute for Healthcare Improvement Global Trigger Tool,^8,9 which is estimated to identify up to 10 times more AEs than other methods, such as voluntary reporting.⁵ This protocol includes 28 triggers in the Cares and Medication Modules that serve as indicators that an AE may have occurred. The presence of a trigger is not necessarily an AE but a clue for further analysis. Two investigators (AS and CS) independently systematically searched for the presence of triggers within each patient chart. Trigger identification prompted in-depth analysis to confirm the occurrence of an AE and to characterize its severity by using the National Coordinating Council for Medication Error Reporting and Prevention categorization.¹⁰ An AE was coded when independent reviewers identified evidence of a preventable or nonpreventable “noxious and unintended event occurring in association with medical care.”⁹ By definition, any AEs identified were patient harms. Findings were reviewed weekly to ensure agreement, and discrepancies were adjudicated by a third investigator (MB).

All study data were collected by using REDCap electronic data capture tools hosted at the University of Washington.¹¹ The University of Washington Institutional Review Board granted approval for this study.

Study Outcome and Statistical Analysis

The primary outcome was AEs per group with results calculated in three ways: AEs per 1,000 patient-days, AEs per 100 admissions, and percent of admissions with an AE. The risk ratio (RR) for the percent of admissions with an AE and the incidence rate ratio (IRR) for AEs per 1,000 patient-days were calculated for the comparison of significance.

Other data were analyzed by using Pearson’s chi square for categorical data, Student t test for normally distributed quantitative data, and Wilcoxon rank-sum (Mann–Whitney) for the length of stay (due to skew). Analyses were conducted using STATA (version 15.1, College Station, TX).

This work follows standards for reporting observational students as outlined in the STROBE statement.¹²

Patient Demographics

Both groups were predominantly white/non-Hispanic, male, and English-speaking (Table 1). More patients without definite medical acuity were covered by public insurance (78.9% vs 69.8%, P = .010) and discharged to homelessness (34.8% vs 22.6%, P = .041).

Measures of Illness

Patients considered definitely medically appropriate had lower ESI scores, indicative of more acute presentation, than those without definite medical acuity (2.73 [95% CI 2.64-2.81] vs 2.87 [95% CI 2.78-2.95], P = .026). There was no difference in CCI scores (Table 1).

Reason for Admission and Outcomes

Admissions considered definitely medically appropriate more frequently had an identified diagnosis/syndrome (66% vs 53%) or objective measurement (8.7% vs 2.7%) listed as the reason for admission, whereas patients admitted without definite medical acuity more freuqently had undifferentiated symptoms (34.7% vs 24%) or other/disposition (6% vs 1.3%) listed. The most common factors that triage physicians cited as contributing to the decision to admit patients without definite medical acuity included homelessness (34%), lack of outpatient social support (32%), and substance use disorder (25%). More details are available in Appendix Tables 1 and 2.

Admissions without definite medical acuity were longer than those with definite medical acuity (6.6 vs 6.0 days, P = .038), but there was no difference in emergency department readmissions within 48 hours or hospital readmissions within 30 days (Table 1).

Adverse Events

We identified 76 AEs in 41 admissions without definite medical acuity (range 0-10 AEs per admission) and 63 AEs in 44 definitely medically appropriate admissions (range 0-4 AEs per admission). The percentage of admissions with AE (27.3% vs 29.3%; RR 0.93, 95% CI 0.65-1.34, P = .70) and the rate of AE/1,000 patient-days (76.8 vs 70.4; IRR 1.09, 95% CI 0.77-1.55, P = .61) did not show statistically significant differences. The distribution of AE severity was similar between the two groups (Table 2). Most identified AEs caused temporary harm to the patient and were rated at severity levels E or F. Severe AEs, including at least one level I (patient death), occurred in both groups. The complete listing of positive triggers leading to adverse event identification by group and severity is available in Appendix Table 3.

By using a robust, standardized method, we found that patients admitted without definite medical acuity experienced the same number of inpatient AEs as patients admitted for definitely medically appropriate reasons. While the groups were relatively similar overall in terms of demographics and chronic comorbidity, we found evidence of social vulnerability in the group admitted without definite medical acuity in the form of increased rates of homelessness, triage physician concern regarding the lack of outpatient social support, and disposition-related reasons for admission. That both groups suffered harm―including patient death―while admitted to the hospital is striking, in particular for those patients who were admitted because of the lack of suitable outpatient options.

The potential limitations to the generalizability of this work include the single-site, safety-net setting and the use of individual physician determination of admission appropriateness. The proportion of admissions without definite medical acuity reported here is similar to that reported by previously published admission decision-making studies,^2,3 and the rate of AEs observed is similar to rates measured in other studies using the trigger tool methodology.^5,13 These similarities suggest some commonality across settings. Our study treats triage physician assessment as the marker of difference in defining the two groups and is an inherently subjective assessment that is reflective of real-world, holistic decision-making. Notably, the triage physician assessment was corroborated by corresponding differences in the ESI score, an acute triage assessment completed by a clinician outside of our team.

This study adds foundational knowledge to the risk/benefit discussion surrounding the decision to admit. Physician admission decisions are likely influenced by concern for the safety of vulnerable patients. Our results suggest that considering the risk of hospitalization itself in this decision-making remains important.

Evidence exists that physicians consider what may be called “social” or “nonmedical” factors (lack of social support or barriers to access) in hospital admission decision-making and that patients are hospitalized even in the absence of a level of medical acuity warranting admission.^1-3 Although hospitalization is associated with the risk of adverse events (AEs),⁴ whether this risk is related to the medical acuity of admission remains unclear. Our study sought to quantify the AEs experienced by patients hospitalized without definite medical acuity compared with those experienced by patients hospitalized with a definite medically appropriate indication for admission.

Setting and Database Used for Analysis

This study was conducted at an urban, safety-net, public teaching hospital. At our site, calls for medical admissions are always answered by a hospital medicine attending physician (“triage physician”) who works collaboratively with the referring physician to facilitate appropriate disposition. Many of these discussions occur via telephone, but the triage physician may also assess the patient directly if needed. This study involved 24 triage physicians who directly assessed the patient in 65% of the cases.

At the time of each admission call, the triage physician logs the following information into a central triage database: date and time of call, patient location, reason for admission, assessment of appropriateness for medical floor, contributing factors to admission decision-making, and patient disposition.

Admission Appropriateness Group Designation

To be considered for inclusion in this study, calls must have originated from the emergency department and resulted in admission to the general medicine floor on either a resident teaching or hospitalist service from February 1, 2018 to June 1, 2018. This time frame was selected to avoid the start of a new academic cycle in late June that may confound AE rates.

The designation of appropriateness was determined by the triage physician’s logged response to triage database questions at the time of the admission call. Of the 748 admissions meeting inclusion criteria, 513 (68.6%) were considered definitely appropriate on the basis of the triage physician’s response to the question “Based ONLY on the medical reason for hospitalization, in your opinion, how appropriate is this admission to the medicine floor service?” Furthermore, 169 (22.6%) were considered without definite medical acuity on the basis of the triage physician’s indication that “severity of medical problems alone may not require inpatient hospitalization” (Appendix Figure 1).

Study Design

Following a retrospective cohort study design, we systematically sampled 150 admissions from those “admitted without definite medical acuity” to create the exposure group and 150 from the “definitely medically appropriate” admissions to create the nonexposure group. Our sampling method involved selecting every third record until reaching the target sample size. This method and group sizes were determined prior to beginning data collection. Given the expected incidence of 33% AEs in the unexposed group (consistent with previous reports of AEs using the trigger tool⁵), we anticipated that a total sample size of 300 would be appropriate to capture a relative risk of at least 1.5 with 80% power and 95% confidence level.

Chart review was performed to capture patient demographics, admission characteristics, and hospitalization outcomes. We captured emergency severity index (ESI)⁶, a validated, reliable triage assessment score assigned by our emergency department, as a measurement of acute illness and calculated the Charlson comorbidity index (CCI)⁷ as a measurement of chronic comorbidity.

Identification of Adverse Events

We measured AEs by using the Institute for Healthcare Improvement Global Trigger Tool,^8,9 which is estimated to identify up to 10 times more AEs than other methods, such as voluntary reporting.⁵ This protocol includes 28 triggers in the Cares and Medication Modules that serve as indicators that an AE may have occurred. The presence of a trigger is not necessarily an AE but a clue for further analysis. Two investigators (AS and CS) independently systematically searched for the presence of triggers within each patient chart. Trigger identification prompted in-depth analysis to confirm the occurrence of an AE and to characterize its severity by using the National Coordinating Council for Medication Error Reporting and Prevention categorization.¹⁰ An AE was coded when independent reviewers identified evidence of a preventable or nonpreventable “noxious and unintended event occurring in association with medical care.”⁹ By definition, any AEs identified were patient harms. Findings were reviewed weekly to ensure agreement, and discrepancies were adjudicated by a third investigator (MB).

All study data were collected by using REDCap electronic data capture tools hosted at the University of Washington.¹¹ The University of Washington Institutional Review Board granted approval for this study.

Study Outcome and Statistical Analysis

The primary outcome was AEs per group with results calculated in three ways: AEs per 1,000 patient-days, AEs per 100 admissions, and percent of admissions with an AE. The risk ratio (RR) for the percent of admissions with an AE and the incidence rate ratio (IRR) for AEs per 1,000 patient-days were calculated for the comparison of significance.

Other data were analyzed by using Pearson’s chi square for categorical data, Student t test for normally distributed quantitative data, and Wilcoxon rank-sum (Mann–Whitney) for the length of stay (due to skew). Analyses were conducted using STATA (version 15.1, College Station, TX).

This work follows standards for reporting observational students as outlined in the STROBE statement.¹²

Patient Demographics

Both groups were predominantly white/non-Hispanic, male, and English-speaking (Table 1). More patients without definite medical acuity were covered by public insurance (78.9% vs 69.8%, P = .010) and discharged to homelessness (34.8% vs 22.6%, P = .041).

Measures of Illness

Patients considered definitely medically appropriate had lower ESI scores, indicative of more acute presentation, than those without definite medical acuity (2.73 [95% CI 2.64-2.81] vs 2.87 [95% CI 2.78-2.95], P = .026). There was no difference in CCI scores (Table 1).

Reason for Admission and Outcomes

Admissions considered definitely medically appropriate more frequently had an identified diagnosis/syndrome (66% vs 53%) or objective measurement (8.7% vs 2.7%) listed as the reason for admission, whereas patients admitted without definite medical acuity more freuqently had undifferentiated symptoms (34.7% vs 24%) or other/disposition (6% vs 1.3%) listed. The most common factors that triage physicians cited as contributing to the decision to admit patients without definite medical acuity included homelessness (34%), lack of outpatient social support (32%), and substance use disorder (25%). More details are available in Appendix Tables 1 and 2.

Admissions without definite medical acuity were longer than those with definite medical acuity (6.6 vs 6.0 days, P = .038), but there was no difference in emergency department readmissions within 48 hours or hospital readmissions within 30 days (Table 1).

Adverse Events

We identified 76 AEs in 41 admissions without definite medical acuity (range 0-10 AEs per admission) and 63 AEs in 44 definitely medically appropriate admissions (range 0-4 AEs per admission). The percentage of admissions with AE (27.3% vs 29.3%; RR 0.93, 95% CI 0.65-1.34, P = .70) and the rate of AE/1,000 patient-days (76.8 vs 70.4; IRR 1.09, 95% CI 0.77-1.55, P = .61) did not show statistically significant differences. The distribution of AE severity was similar between the two groups (Table 2). Most identified AEs caused temporary harm to the patient and were rated at severity levels E or F. Severe AEs, including at least one level I (patient death), occurred in both groups. The complete listing of positive triggers leading to adverse event identification by group and severity is available in Appendix Table 3.

By using a robust, standardized method, we found that patients admitted without definite medical acuity experienced the same number of inpatient AEs as patients admitted for definitely medically appropriate reasons. While the groups were relatively similar overall in terms of demographics and chronic comorbidity, we found evidence of social vulnerability in the group admitted without definite medical acuity in the form of increased rates of homelessness, triage physician concern regarding the lack of outpatient social support, and disposition-related reasons for admission. That both groups suffered harm―including patient death―while admitted to the hospital is striking, in particular for those patients who were admitted because of the lack of suitable outpatient options.

The potential limitations to the generalizability of this work include the single-site, safety-net setting and the use of individual physician determination of admission appropriateness. The proportion of admissions without definite medical acuity reported here is similar to that reported by previously published admission decision-making studies,^2,3 and the rate of AEs observed is similar to rates measured in other studies using the trigger tool methodology.^5,13 These similarities suggest some commonality across settings. Our study treats triage physician assessment as the marker of difference in defining the two groups and is an inherently subjective assessment that is reflective of real-world, holistic decision-making. Notably, the triage physician assessment was corroborated by corresponding differences in the ESI score, an acute triage assessment completed by a clinician outside of our team.

This study adds foundational knowledge to the risk/benefit discussion surrounding the decision to admit. Physician admission decisions are likely influenced by concern for the safety of vulnerable patients. Our results suggest that considering the risk of hospitalization itself in this decision-making remains important.

As you review the electronic health record before rounds in the morning, you notice a red exclamation mark in the chart of a patient who was admitted two days ago for an acute chronic obstructive pulmonary disease (COPD) exacerbation. The patient’s respiratory rate (RR) this morning is recorded at 24 breaths per minute (bpm). His RR last evening was 16 bpm and he remains on two liters per minute of supplemental oxygen. No one has notified you that he is getting worse, but you stop by the room to confirm that he is clinically stable.

During rounds, the resident states “The respiratory rate is recorded as 24 bpm, which is high, but I never trust the respiratory rate.” You silently agree and confirm your mistrust of the recorded RR.

Elevated RR has been associated with numerous poor outcomes, including mortality after myocardial infarction¹ and death and readmission after acute COPD exacerbation.² Furthermore, RR is used in models to predict mortality and intensive care unit admission,³ as well as in models to identify and predict mortality from sepsis.⁴ Recorded RRs are frequency inaccurate,⁵ and medical staff lack confidence in recorded RR values.⁶ Based on this evidence, you feel justified in your mistrust of recorded RR values. You might even believe that until a high-tech RR monitoring system is invented and implemented at your hospital, human error will forever prevent you from knowing your patients’ true RRs.

However, there is hope. In this issue of the Journal of Hospital Medicine, Keshvani et al.⁷ describe a successful quality improvement project where they employed plan–do–study–act methodology in a single inpatient unit to improve the accuracy of recorded RR. Before their project, only 36% of RR measurements were accurate, and there was considerable heterogeneity in the RR measurement technique. To address this problem, an interdisciplinary team of patient care assistants (PCAs), nurses, physicians, and hospital administration developed a plan to identify barriers, improve workflow, and educate stakeholders in RR recording.

The authors created a low-cost, “low-tech” intervention that consisted of training and educating PCAs on the correct technique and the importance of RR measurement, modifying workflow to incorporate RR measurement into a 30-second period of automated blood pressure measurement, and adding stopwatches to the vital sign carts. The RR measurements obtained by PCAs were compared with the RR measurements obtained by trained team members to assess for accuracy. PCA-obtained RR measurements were also compared with two control units, both before and after the intervention. Secondary outcomes included time to complete vital sign measurements and the incidence of systemic inflammatory response syndrome (SIRS) specifically due to tachypnea. The authors hypothesized that improved RR accuracy would reduce the number of falsely elevated RRs and could reduce the rate of SIRS.

The intervention improved the accuracy of PCA-obtained RRs from 36% to 58% and decreased the median RR from 18 to 14 breaths per minute. The implementation also resulted in a more normal distribution of RR in the intervention unit compared with the control unit. Interestingly, this intervention did not increase the time spent in obtaining vital signs—in fact, the time to complete vital signs decreased from a median of 2:26 to 1:55 minutes. In addition, tachypnea-specific SIRS incidence was reduced by 7.8% per hospitalization. An important implication of this finding is that reducing the false-positive rate of SIRS could possibly decrease unnecessary testing, medical interventions, and alert fatigue.

This project shows that meaningful interventions need not be expensive or overly technologic to have very real clinical effects. It would be very easy for a system to advocate for funding to purchase advanced monitors that purport to remove human error from the situation rather than trying first to improve human performance. Certainly, there is a role for advanced technologies—but improvement need not wait for, or be completely predicated on, these new technologies. The first barrier often expressed when evaluating a potential improvement initiative is that “we don’t have time for that”. This project demonstrates that innovations to improve care can also benefit the care team and improve workflow. Certainly, this project is not definitive and should be replicated elsewhere, but it is an important first step.

In an era where technology is expanding rapidly and the pace of innovation is breathtaking, we have an obligation to ensure that we are getting the basics right. Further, we must not take core tasks—such as vital signs, physical examination, and medication reconciliation—for granted, nor should we accept that they are as they will be. We discuss and debate the merits of advanced imaging, artificial intelligence, and machine learning­—which are certainly exciting advances—but we must occasionally pause, breathe, and examine our practice to make sure that we do not overlook things that are truly vital to our patients’ care.

Disclosures

The authors have nothing to disclose.

As you review the electronic health record before rounds in the morning, you notice a red exclamation mark in the chart of a patient who was admitted two days ago for an acute chronic obstructive pulmonary disease (COPD) exacerbation. The patient’s respiratory rate (RR) this morning is recorded at 24 breaths per minute (bpm). His RR last evening was 16 bpm and he remains on two liters per minute of supplemental oxygen. No one has notified you that he is getting worse, but you stop by the room to confirm that he is clinically stable.

During rounds, the resident states “The respiratory rate is recorded as 24 bpm, which is high, but I never trust the respiratory rate.” You silently agree and confirm your mistrust of the recorded RR.

Elevated RR has been associated with numerous poor outcomes, including mortality after myocardial infarction¹ and death and readmission after acute COPD exacerbation.² Furthermore, RR is used in models to predict mortality and intensive care unit admission,³ as well as in models to identify and predict mortality from sepsis.⁴ Recorded RRs are frequency inaccurate,⁵ and medical staff lack confidence in recorded RR values.⁶ Based on this evidence, you feel justified in your mistrust of recorded RR values. You might even believe that until a high-tech RR monitoring system is invented and implemented at your hospital, human error will forever prevent you from knowing your patients’ true RRs.

However, there is hope. In this issue of the Journal of Hospital Medicine, Keshvani et al.⁷ describe a successful quality improvement project where they employed plan–do–study–act methodology in a single inpatient unit to improve the accuracy of recorded RR. Before their project, only 36% of RR measurements were accurate, and there was considerable heterogeneity in the RR measurement technique. To address this problem, an interdisciplinary team of patient care assistants (PCAs), nurses, physicians, and hospital administration developed a plan to identify barriers, improve workflow, and educate stakeholders in RR recording.

The authors created a low-cost, “low-tech” intervention that consisted of training and educating PCAs on the correct technique and the importance of RR measurement, modifying workflow to incorporate RR measurement into a 30-second period of automated blood pressure measurement, and adding stopwatches to the vital sign carts. The RR measurements obtained by PCAs were compared with the RR measurements obtained by trained team members to assess for accuracy. PCA-obtained RR measurements were also compared with two control units, both before and after the intervention. Secondary outcomes included time to complete vital sign measurements and the incidence of systemic inflammatory response syndrome (SIRS) specifically due to tachypnea. The authors hypothesized that improved RR accuracy would reduce the number of falsely elevated RRs and could reduce the rate of SIRS.

The intervention improved the accuracy of PCA-obtained RRs from 36% to 58% and decreased the median RR from 18 to 14 breaths per minute. The implementation also resulted in a more normal distribution of RR in the intervention unit compared with the control unit. Interestingly, this intervention did not increase the time spent in obtaining vital signs—in fact, the time to complete vital signs decreased from a median of 2:26 to 1:55 minutes. In addition, tachypnea-specific SIRS incidence was reduced by 7.8% per hospitalization. An important implication of this finding is that reducing the false-positive rate of SIRS could possibly decrease unnecessary testing, medical interventions, and alert fatigue.

This project shows that meaningful interventions need not be expensive or overly technologic to have very real clinical effects. It would be very easy for a system to advocate for funding to purchase advanced monitors that purport to remove human error from the situation rather than trying first to improve human performance. Certainly, there is a role for advanced technologies—but improvement need not wait for, or be completely predicated on, these new technologies. The first barrier often expressed when evaluating a potential improvement initiative is that “we don’t have time for that”. This project demonstrates that innovations to improve care can also benefit the care team and improve workflow. Certainly, this project is not definitive and should be replicated elsewhere, but it is an important first step.

In an era where technology is expanding rapidly and the pace of innovation is breathtaking, we have an obligation to ensure that we are getting the basics right. Further, we must not take core tasks—such as vital signs, physical examination, and medication reconciliation—for granted, nor should we accept that they are as they will be. We discuss and debate the merits of advanced imaging, artificial intelligence, and machine learning­—which are certainly exciting advances—but we must occasionally pause, breathe, and examine our practice to make sure that we do not overlook things that are truly vital to our patients’ care.

Disclosures

The authors have nothing to disclose.

As you review the electronic health record before rounds in the morning, you notice a red exclamation mark in the chart of a patient who was admitted two days ago for an acute chronic obstructive pulmonary disease (COPD) exacerbation. The patient’s respiratory rate (RR) this morning is recorded at 24 breaths per minute (bpm). His RR last evening was 16 bpm and he remains on two liters per minute of supplemental oxygen. No one has notified you that he is getting worse, but you stop by the room to confirm that he is clinically stable.

During rounds, the resident states “The respiratory rate is recorded as 24 bpm, which is high, but I never trust the respiratory rate.” You silently agree and confirm your mistrust of the recorded RR.

Elevated RR has been associated with numerous poor outcomes, including mortality after myocardial infarction¹ and death and readmission after acute COPD exacerbation.² Furthermore, RR is used in models to predict mortality and intensive care unit admission,³ as well as in models to identify and predict mortality from sepsis.⁴ Recorded RRs are frequency inaccurate,⁵ and medical staff lack confidence in recorded RR values.⁶ Based on this evidence, you feel justified in your mistrust of recorded RR values. You might even believe that until a high-tech RR monitoring system is invented and implemented at your hospital, human error will forever prevent you from knowing your patients’ true RRs.

However, there is hope. In this issue of the Journal of Hospital Medicine, Keshvani et al.⁷ describe a successful quality improvement project where they employed plan–do–study–act methodology in a single inpatient unit to improve the accuracy of recorded RR. Before their project, only 36% of RR measurements were accurate, and there was considerable heterogeneity in the RR measurement technique. To address this problem, an interdisciplinary team of patient care assistants (PCAs), nurses, physicians, and hospital administration developed a plan to identify barriers, improve workflow, and educate stakeholders in RR recording.

The authors created a low-cost, “low-tech” intervention that consisted of training and educating PCAs on the correct technique and the importance of RR measurement, modifying workflow to incorporate RR measurement into a 30-second period of automated blood pressure measurement, and adding stopwatches to the vital sign carts. The RR measurements obtained by PCAs were compared with the RR measurements obtained by trained team members to assess for accuracy. PCA-obtained RR measurements were also compared with two control units, both before and after the intervention. Secondary outcomes included time to complete vital sign measurements and the incidence of systemic inflammatory response syndrome (SIRS) specifically due to tachypnea. The authors hypothesized that improved RR accuracy would reduce the number of falsely elevated RRs and could reduce the rate of SIRS.

The intervention improved the accuracy of PCA-obtained RRs from 36% to 58% and decreased the median RR from 18 to 14 breaths per minute. The implementation also resulted in a more normal distribution of RR in the intervention unit compared with the control unit. Interestingly, this intervention did not increase the time spent in obtaining vital signs—in fact, the time to complete vital signs decreased from a median of 2:26 to 1:55 minutes. In addition, tachypnea-specific SIRS incidence was reduced by 7.8% per hospitalization. An important implication of this finding is that reducing the false-positive rate of SIRS could possibly decrease unnecessary testing, medical interventions, and alert fatigue.

This project shows that meaningful interventions need not be expensive or overly technologic to have very real clinical effects. It would be very easy for a system to advocate for funding to purchase advanced monitors that purport to remove human error from the situation rather than trying first to improve human performance. Certainly, there is a role for advanced technologies—but improvement need not wait for, or be completely predicated on, these new technologies. The first barrier often expressed when evaluating a potential improvement initiative is that “we don’t have time for that”. This project demonstrates that innovations to improve care can also benefit the care team and improve workflow. Certainly, this project is not definitive and should be replicated elsewhere, but it is an important first step.

In an era where technology is expanding rapidly and the pace of innovation is breathtaking, we have an obligation to ensure that we are getting the basics right. Further, we must not take core tasks—such as vital signs, physical examination, and medication reconciliation—for granted, nor should we accept that they are as they will be. We discuss and debate the merits of advanced imaging, artificial intelligence, and machine learning­—which are certainly exciting advances—but we must occasionally pause, breathe, and examine our practice to make sure that we do not overlook things that are truly vital to our patients’ care.

Disclosures

The authors have nothing to disclose.

The past 20 years has seen an explosion of approaches to improve the recognition of children who deteriorate in the hospital. Early Warning Scores, Rapid Response Teams, Situational Awareness, and Parent-Triggered Activation systems are a few of the safety initiatives implemented worldwide. Many have an inherent face validity; for example, it would appear to be intuitive that highlighting the changes in physiology via a Pediatric Early Warning Score (PEWS) would enable staff to recognize a change in disease process and intervene accordingly. However, although mortality trends have been shown to diminish over time,¹ the evidence base supporting their impact has often been quite heterogeneous.^2,3 In particular, a recent international randomized control trial of a PEWS approach was found not to improve overall mortality.⁴

A major challenge with the evaluation of these patient safety systems is the reliance on mortality as an outcome measure. This is relatively rare, even in large tertiary institutions with complex patients and finding other proxy measures of quality of care are important. Hussain et al. have created a relatively easy to measure metric, an emergency transfer (ET). The benefit of the ET is its simplicity and transferability, which is described as follows:

“Emergency Transfer (ET) is defined as any patient transferred to the ICU where the patient received intubation, inotropes, or three or more fluid boluses in the first hour after arrival or before transfer.”⁵

All these components are easily extractable from written or electronic records and are representative of meaningful deterioration. Pressure on bed states, challenges with staff skill mix, and increasing parental expectation may all impact on decisions to transfer patients. The ET metric is relatively immune to these biases as its tight time definition separates it from the previous Bonafide et al.⁶ measure (similar interventions but within a 12-hour window) as being centered on an abrupt critical change, rather than a potential drift toward deterioration. This makes the measure useful not only to an individual institution to measure the impact of an intervention but also internationally, as a comparison between institutions will not be influenced by health system differences.

The ET metric is important as Hussain et al. have demonstrated that it is associated with a worse outcome for the child both as a concrete outcome (increased mortality when it does occur) and as an experience (a longer stay in hospital). “You can’t improve what you can’t measure” is an old improvement maxim, and only by broadening our use of alternative metrics of care will we be able to understand which interventions will make a difference to patients. Certainly, evidence suggests that cultures, hierarchies, and leadership may well be as important as other more concrete or tangible tools,⁷ but these have seldom been evaluated as part of studies on improving the response to deterioration. The pediatric early warning system utilization and mortality avoidance (PUMA) study, a research program funded by the National Institute for Health Research (United Kingdom), is exploring these tools and will likely report later in 2019.⁸

Two immediate practical implications of this work emerge, which should be of relevance to clinical leaders in children’s hospitals. The first is that it is highly likely that there will be some events you cannot anticipate. A bronchiolitic infant is always likely to suddenly plug off, and invasive group A streptococcus is a mastery of mimicry and deceit. The authors noted that even with a mature, long-standing Rapid Response System process, ETs were still associated with adverse outcomes. Therefore, it may well be that the ET metric measured over time delineates a locally defined acceptable level of unplanned intensive care admission. If your hospital is significantly above this, they must seriously look at how they can improve their performance. It should be noted here that there were only 45 ETs identified in 4.5 years in Cincinnati and 50% of these were from specialist units within the hospital. It is possible that perhaps the ETs will in the future become as rare as mortality is today, and as hospitals improve, new frames of reference will be needed.

These new references are likely to come from high-performing child health institutions such as those in Philadelphia and Cincinnati, and this leads to a second important principle that hospitals should acknowledge. One of the reasons for patient safety success is the relentless pursuit of excellence. The very act of consistently, and transparently, auditing and analyzing performance is vital to change outcomes. We should digest, evaluate, adopt, and improve the research that groups such as these are undertaking as, although sometimes imperfect, they should also inspire us to ensure that children in our own institutions are as safe as they possibly can be.

Disclosure

Dr. Roland reports that he is currently the cochief investigator of a National Institute for Health Research (NIHR) grant investigating pediatric early warning systems (the PUMA study)

The past 20 years has seen an explosion of approaches to improve the recognition of children who deteriorate in the hospital. Early Warning Scores, Rapid Response Teams, Situational Awareness, and Parent-Triggered Activation systems are a few of the safety initiatives implemented worldwide. Many have an inherent face validity; for example, it would appear to be intuitive that highlighting the changes in physiology via a Pediatric Early Warning Score (PEWS) would enable staff to recognize a change in disease process and intervene accordingly. However, although mortality trends have been shown to diminish over time,¹ the evidence base supporting their impact has often been quite heterogeneous.^2,3 In particular, a recent international randomized control trial of a PEWS approach was found not to improve overall mortality.⁴

A major challenge with the evaluation of these patient safety systems is the reliance on mortality as an outcome measure. This is relatively rare, even in large tertiary institutions with complex patients and finding other proxy measures of quality of care are important. Hussain et al. have created a relatively easy to measure metric, an emergency transfer (ET). The benefit of the ET is its simplicity and transferability, which is described as follows:

“Emergency Transfer (ET) is defined as any patient transferred to the ICU where the patient received intubation, inotropes, or three or more fluid boluses in the first hour after arrival or before transfer.”⁵

All these components are easily extractable from written or electronic records and are representative of meaningful deterioration. Pressure on bed states, challenges with staff skill mix, and increasing parental expectation may all impact on decisions to transfer patients. The ET metric is relatively immune to these biases as its tight time definition separates it from the previous Bonafide et al.⁶ measure (similar interventions but within a 12-hour window) as being centered on an abrupt critical change, rather than a potential drift toward deterioration. This makes the measure useful not only to an individual institution to measure the impact of an intervention but also internationally, as a comparison between institutions will not be influenced by health system differences.

The ET metric is important as Hussain et al. have demonstrated that it is associated with a worse outcome for the child both as a concrete outcome (increased mortality when it does occur) and as an experience (a longer stay in hospital). “You can’t improve what you can’t measure” is an old improvement maxim, and only by broadening our use of alternative metrics of care will we be able to understand which interventions will make a difference to patients. Certainly, evidence suggests that cultures, hierarchies, and leadership may well be as important as other more concrete or tangible tools,⁷ but these have seldom been evaluated as part of studies on improving the response to deterioration. The pediatric early warning system utilization and mortality avoidance (PUMA) study, a research program funded by the National Institute for Health Research (United Kingdom), is exploring these tools and will likely report later in 2019.⁸

Two immediate practical implications of this work emerge, which should be of relevance to clinical leaders in children’s hospitals. The first is that it is highly likely that there will be some events you cannot anticipate. A bronchiolitic infant is always likely to suddenly plug off, and invasive group A streptococcus is a mastery of mimicry and deceit. The authors noted that even with a mature, long-standing Rapid Response System process, ETs were still associated with adverse outcomes. Therefore, it may well be that the ET metric measured over time delineates a locally defined acceptable level of unplanned intensive care admission. If your hospital is significantly above this, they must seriously look at how they can improve their performance. It should be noted here that there were only 45 ETs identified in 4.5 years in Cincinnati and 50% of these were from specialist units within the hospital. It is possible that perhaps the ETs will in the future become as rare as mortality is today, and as hospitals improve, new frames of reference will be needed.

These new references are likely to come from high-performing child health institutions such as those in Philadelphia and Cincinnati, and this leads to a second important principle that hospitals should acknowledge. One of the reasons for patient safety success is the relentless pursuit of excellence. The very act of consistently, and transparently, auditing and analyzing performance is vital to change outcomes. We should digest, evaluate, adopt, and improve the research that groups such as these are undertaking as, although sometimes imperfect, they should also inspire us to ensure that children in our own institutions are as safe as they possibly can be.

Disclosure

Dr. Roland reports that he is currently the cochief investigator of a National Institute for Health Research (NIHR) grant investigating pediatric early warning systems (the PUMA study)

The past 20 years has seen an explosion of approaches to improve the recognition of children who deteriorate in the hospital. Early Warning Scores, Rapid Response Teams, Situational Awareness, and Parent-Triggered Activation systems are a few of the safety initiatives implemented worldwide. Many have an inherent face validity; for example, it would appear to be intuitive that highlighting the changes in physiology via a Pediatric Early Warning Score (PEWS) would enable staff to recognize a change in disease process and intervene accordingly. However, although mortality trends have been shown to diminish over time,¹ the evidence base supporting their impact has often been quite heterogeneous.^2,3 In particular, a recent international randomized control trial of a PEWS approach was found not to improve overall mortality.⁴

A major challenge with the evaluation of these patient safety systems is the reliance on mortality as an outcome measure. This is relatively rare, even in large tertiary institutions with complex patients and finding other proxy measures of quality of care are important. Hussain et al. have created a relatively easy to measure metric, an emergency transfer (ET). The benefit of the ET is its simplicity and transferability, which is described as follows:

“Emergency Transfer (ET) is defined as any patient transferred to the ICU where the patient received intubation, inotropes, or three or more fluid boluses in the first hour after arrival or before transfer.”⁵

All these components are easily extractable from written or electronic records and are representative of meaningful deterioration. Pressure on bed states, challenges with staff skill mix, and increasing parental expectation may all impact on decisions to transfer patients. The ET metric is relatively immune to these biases as its tight time definition separates it from the previous Bonafide et al.⁶ measure (similar interventions but within a 12-hour window) as being centered on an abrupt critical change, rather than a potential drift toward deterioration. This makes the measure useful not only to an individual institution to measure the impact of an intervention but also internationally, as a comparison between institutions will not be influenced by health system differences.

The ET metric is important as Hussain et al. have demonstrated that it is associated with a worse outcome for the child both as a concrete outcome (increased mortality when it does occur) and as an experience (a longer stay in hospital). “You can’t improve what you can’t measure” is an old improvement maxim, and only by broadening our use of alternative metrics of care will we be able to understand which interventions will make a difference to patients. Certainly, evidence suggests that cultures, hierarchies, and leadership may well be as important as other more concrete or tangible tools,⁷ but these have seldom been evaluated as part of studies on improving the response to deterioration. The pediatric early warning system utilization and mortality avoidance (PUMA) study, a research program funded by the National Institute for Health Research (United Kingdom), is exploring these tools and will likely report later in 2019.⁸

Two immediate practical implications of this work emerge, which should be of relevance to clinical leaders in children’s hospitals. The first is that it is highly likely that there will be some events you cannot anticipate. A bronchiolitic infant is always likely to suddenly plug off, and invasive group A streptococcus is a mastery of mimicry and deceit. The authors noted that even with a mature, long-standing Rapid Response System process, ETs were still associated with adverse outcomes. Therefore, it may well be that the ET metric measured over time delineates a locally defined acceptable level of unplanned intensive care admission. If your hospital is significantly above this, they must seriously look at how they can improve their performance. It should be noted here that there were only 45 ETs identified in 4.5 years in Cincinnati and 50% of these were from specialist units within the hospital. It is possible that perhaps the ETs will in the future become as rare as mortality is today, and as hospitals improve, new frames of reference will be needed.

These new references are likely to come from high-performing child health institutions such as those in Philadelphia and Cincinnati, and this leads to a second important principle that hospitals should acknowledge. One of the reasons for patient safety success is the relentless pursuit of excellence. The very act of consistently, and transparently, auditing and analyzing performance is vital to change outcomes. We should digest, evaluate, adopt, and improve the research that groups such as these are undertaking as, although sometimes imperfect, they should also inspire us to ensure that children in our own institutions are as safe as they possibly can be.

Disclosure

Dr. Roland reports that he is currently the cochief investigator of a National Institute for Health Research (NIHR) grant investigating pediatric early warning systems (the PUMA study)