Hospital Medicine

In this issue of the Journal of Hospital Medicine, McAlister et al.¹ compared the ability of the Clinical Frailty Scale (CFS) and the Hospital Frailty Risk Score (HFRS) to predict 30-day readmission or death. The authors prospectively assessed adult patients aged ≥18 years without cognitive impairment being discharged back to the community after medical admissions. They demonstrated only modest overlap in frailty designation between HFRS and CFS and concluded that CFS is better than HFRS for predicting the outcomes of interest.

Before a prediction rule is widely adopted for use in routine practice, robust external validation is needed.² Factors such as the prevalence of disease in a population, the clinical competencies of a health system, the socioeconomic status, and the ethnicity of the population can all affect how well a clinical rule performs, but may not become apparent until a prospective validation in a different population is attempted.

In developing the HFRS, Gilbert et al. aimed to create a low-cost, highly generalizable method of identifying frailty using International Classification of Diseases (ICD) 10 billing codes.³ The derivation and validation cohorts for HFRS included older adults aged >75 years in the United Kingdom, many of whom had cognitive impairment. Therefore, it is not surprising that the tool behaved very differently in the younger Canadian cohort described by McAlister et al. where persons with cognitive impairment were excluded. That the HFRS had less predictability in the Canadian cohort may simply indicate that it performs better in an older population with cognitive vulnerabilities; given the frailty constructs of the CFS, it may provide less insights in older populations.

We applaud the efforts to find a way to better identify high-risk groups of adults. We also appreciate the increasing attention to function and other frailty-related domains in risk prediction models. Nevertheless, we recommend caution in using any of the many existing frailty indices⁴ in risk prediction tools unless it is clear what domains of frailty are most relevant for the predicted outcome and what population is the subject of interest.

One of the challenges of choosing an appropriate frailty tool is that different tools are measuring different domains or constructs of frailty. Most consider frailty either as a physical phenotype⁵ or as a more multifaceted construct with impairments in physical and mental health, function, and social interaction.⁶ There is often poor overlap between those individuals identified as frail by different measures, highlighting that they are in fact identifying different people within the population studied and have different predictive abilities.

An ideal frailty tool for clinical use would allow clinicians to identify high-risk patients relative to specific outcome(s) in real time prior to discharge from hospital or prior to a sentinel event in the community. CFS can be calculated at the bedside, but HFRS calculation can only be done retrospectively when medical records are coded for claims after discharge. This makes HFRS more suited to research or post hoc quality measure work and CFS more suited to clinical use as the authors describe.

Although using a frailty indicator to help determine those at high risk of early readmission is an important objective, the presence of frailty accounts for only part of a person’s risk for readmission or other untoward events. Reasons for readmissions are complex and often heavily weighted on a lack of social and community supports. A deeper understanding of the reasons for readmission is needed to establish whether readmission of these complex patients has more to do with frailty or other drivers such as poor transitions of care.

The prevalence of frailty will continue to increase as our population ages. Definitions of frailty vary, but there is a broad agreement that frailty, regardless of how it is constructed, increases with age, results in multisystem changes, and leads to increased healthcare utilization and costs. Preventing the development of frailty, identifying frailty, and developing interventions to address frailty in and out of the hospital setting are all vital. We welcome further research regarding the biopsychosocial constructs of frailty, how they overlap with the frailty phenotype, and how these constructs inform both our understanding of frailty and the use of frailty tools.

Disclosures

The authors have no conflicts of interest to report.

In this issue of the Journal of Hospital Medicine, McAlister et al.¹ compared the ability of the Clinical Frailty Scale (CFS) and the Hospital Frailty Risk Score (HFRS) to predict 30-day readmission or death. The authors prospectively assessed adult patients aged ≥18 years without cognitive impairment being discharged back to the community after medical admissions. They demonstrated only modest overlap in frailty designation between HFRS and CFS and concluded that CFS is better than HFRS for predicting the outcomes of interest.

Before a prediction rule is widely adopted for use in routine practice, robust external validation is needed.² Factors such as the prevalence of disease in a population, the clinical competencies of a health system, the socioeconomic status, and the ethnicity of the population can all affect how well a clinical rule performs, but may not become apparent until a prospective validation in a different population is attempted.

In developing the HFRS, Gilbert et al. aimed to create a low-cost, highly generalizable method of identifying frailty using International Classification of Diseases (ICD) 10 billing codes.³ The derivation and validation cohorts for HFRS included older adults aged >75 years in the United Kingdom, many of whom had cognitive impairment. Therefore, it is not surprising that the tool behaved very differently in the younger Canadian cohort described by McAlister et al. where persons with cognitive impairment were excluded. That the HFRS had less predictability in the Canadian cohort may simply indicate that it performs better in an older population with cognitive vulnerabilities; given the frailty constructs of the CFS, it may provide less insights in older populations.

We applaud the efforts to find a way to better identify high-risk groups of adults. We also appreciate the increasing attention to function and other frailty-related domains in risk prediction models. Nevertheless, we recommend caution in using any of the many existing frailty indices⁴ in risk prediction tools unless it is clear what domains of frailty are most relevant for the predicted outcome and what population is the subject of interest.

One of the challenges of choosing an appropriate frailty tool is that different tools are measuring different domains or constructs of frailty. Most consider frailty either as a physical phenotype⁵ or as a more multifaceted construct with impairments in physical and mental health, function, and social interaction.⁶ There is often poor overlap between those individuals identified as frail by different measures, highlighting that they are in fact identifying different people within the population studied and have different predictive abilities.

An ideal frailty tool for clinical use would allow clinicians to identify high-risk patients relative to specific outcome(s) in real time prior to discharge from hospital or prior to a sentinel event in the community. CFS can be calculated at the bedside, but HFRS calculation can only be done retrospectively when medical records are coded for claims after discharge. This makes HFRS more suited to research or post hoc quality measure work and CFS more suited to clinical use as the authors describe.

Although using a frailty indicator to help determine those at high risk of early readmission is an important objective, the presence of frailty accounts for only part of a person’s risk for readmission or other untoward events. Reasons for readmissions are complex and often heavily weighted on a lack of social and community supports. A deeper understanding of the reasons for readmission is needed to establish whether readmission of these complex patients has more to do with frailty or other drivers such as poor transitions of care.

The prevalence of frailty will continue to increase as our population ages. Definitions of frailty vary, but there is a broad agreement that frailty, regardless of how it is constructed, increases with age, results in multisystem changes, and leads to increased healthcare utilization and costs. Preventing the development of frailty, identifying frailty, and developing interventions to address frailty in and out of the hospital setting are all vital. We welcome further research regarding the biopsychosocial constructs of frailty, how they overlap with the frailty phenotype, and how these constructs inform both our understanding of frailty and the use of frailty tools.

Disclosures

The authors have no conflicts of interest to report.

In this issue of the Journal of Hospital Medicine, McAlister et al.¹ compared the ability of the Clinical Frailty Scale (CFS) and the Hospital Frailty Risk Score (HFRS) to predict 30-day readmission or death. The authors prospectively assessed adult patients aged ≥18 years without cognitive impairment being discharged back to the community after medical admissions. They demonstrated only modest overlap in frailty designation between HFRS and CFS and concluded that CFS is better than HFRS for predicting the outcomes of interest.

Before a prediction rule is widely adopted for use in routine practice, robust external validation is needed.² Factors such as the prevalence of disease in a population, the clinical competencies of a health system, the socioeconomic status, and the ethnicity of the population can all affect how well a clinical rule performs, but may not become apparent until a prospective validation in a different population is attempted.

In developing the HFRS, Gilbert et al. aimed to create a low-cost, highly generalizable method of identifying frailty using International Classification of Diseases (ICD) 10 billing codes.³ The derivation and validation cohorts for HFRS included older adults aged >75 years in the United Kingdom, many of whom had cognitive impairment. Therefore, it is not surprising that the tool behaved very differently in the younger Canadian cohort described by McAlister et al. where persons with cognitive impairment were excluded. That the HFRS had less predictability in the Canadian cohort may simply indicate that it performs better in an older population with cognitive vulnerabilities; given the frailty constructs of the CFS, it may provide less insights in older populations.

We applaud the efforts to find a way to better identify high-risk groups of adults. We also appreciate the increasing attention to function and other frailty-related domains in risk prediction models. Nevertheless, we recommend caution in using any of the many existing frailty indices⁴ in risk prediction tools unless it is clear what domains of frailty are most relevant for the predicted outcome and what population is the subject of interest.

One of the challenges of choosing an appropriate frailty tool is that different tools are measuring different domains or constructs of frailty. Most consider frailty either as a physical phenotype⁵ or as a more multifaceted construct with impairments in physical and mental health, function, and social interaction.⁶ There is often poor overlap between those individuals identified as frail by different measures, highlighting that they are in fact identifying different people within the population studied and have different predictive abilities.

An ideal frailty tool for clinical use would allow clinicians to identify high-risk patients relative to specific outcome(s) in real time prior to discharge from hospital or prior to a sentinel event in the community. CFS can be calculated at the bedside, but HFRS calculation can only be done retrospectively when medical records are coded for claims after discharge. This makes HFRS more suited to research or post hoc quality measure work and CFS more suited to clinical use as the authors describe.

Although using a frailty indicator to help determine those at high risk of early readmission is an important objective, the presence of frailty accounts for only part of a person’s risk for readmission or other untoward events. Reasons for readmissions are complex and often heavily weighted on a lack of social and community supports. A deeper understanding of the reasons for readmission is needed to establish whether readmission of these complex patients has more to do with frailty or other drivers such as poor transitions of care.

The prevalence of frailty will continue to increase as our population ages. Definitions of frailty vary, but there is a broad agreement that frailty, regardless of how it is constructed, increases with age, results in multisystem changes, and leads to increased healthcare utilization and costs. Preventing the development of frailty, identifying frailty, and developing interventions to address frailty in and out of the hospital setting are all vital. We welcome further research regarding the biopsychosocial constructs of frailty, how they overlap with the frailty phenotype, and how these constructs inform both our understanding of frailty and the use of frailty tools.

Disclosures

The authors have no conflicts of interest to report.

Anticoagulant use to prevent ischemic strokes in patients with atrial fibrillation (AF) continues to be one of the most challenging decisions facing patients and their physicians, in large part due to significant patient-to-patient variation in both AF-related stroke risk and anticoagulant-associated hemorrhage risk. Now, add a layer of complexity—.how should one approach anticoagulant use following an adverse event such as an acute upper gastrointestinal (GI) hemorrhage? On the one side, the risk of ischemic stroke, and on the other, the risk of recurrent bleeding, either of which can lead to death or disability. Making this decision requires humility, clinical acumen, shared decision-making, and data.

Data on this subject are sparse.^1,2 Observational studies show that patients who restart anticoagulants after GI hemorrhage experience fewer ischemic strokes. These studies also show that patients who restart anticoagulant therapy are healthier than those who do not—in measurable ways and, importantly, in unmeasurable ways. Thus far, observational studies have not sufficiently dealt with confounding by indication; that is, patients who restart anticoagulants are fundamentally different than patients who do not.

In this issue of the Journal of Hospital Medicine^®, Pappas et al. focus on the optimal timing of resuming oral anticoagulation in patients who have sustained acute upper GI bleeds while receiving oral anticoagulation for AF.³ They use a microsimulation modeling approach to address this question, by creating a synthetic population of patients reflective of age, gender, and comorbidities in a United States population of patients with AF. Using data from epidemiologic studies that describe the risk of rebleeding, hemorrhagic complications, and ischemic stroke as well as the quality of life associated with each of these events, the authors have constructed a decision analytic model to determine the optimal day to restart anticoagulation. This modeling approach mitigates confounding by indication, a limitation of observational studies. They report that the optimal day to restart anticoagulant therapy is in the range of 32-51 days. As one would predict, when using direct-acting anticoagulants and for patients with high stroke risk, the investigators find that restarting therapy earlier is associated with greater benefit. These findings help to untangle a knot of risk and benefits facing patients with AF following an acute GI hemorrhage.

Interpreting the results relies on an understanding of the strengths and weaknesses of simulation modeling and the data used in the analysis. Like any research method, the devil is in the details. Stitching together event rates and outcomes from multiple studies, the results of a simulation model are only as good as the studies the model draws from. In particular, assumptions regarding the time-dependent decline in rebleeding risk are a critical component of determining the optimal time to resume anticoagulation. The authors had to make multiple assumptions to project the 24-hour risk of rebleeding determined from the Rockall score to estimate the risk of rebleeding over the next days to months.⁴ Consequently, the results are likely overly precise. Practically, 30-50 days or four to eight weeks may better reflect the precision of the study findings.

Results on optimal timing of resuming anticoagulation therapy are most applicable for patients when the decision to restart anticoagulants has already been made. We part ways with the authors in their conclusion that these results confirm that anticoagulants should be restarted. There are multiple appropriate reasons why anticoagulant therapy should not be restarted following an acute upper GI hemorrhage. For example, in observational studies, patients not restarted on anticoagulant therapy were more likely to have a history of falls and to have had severe bleeds.¹ Furthermore, patients who do not restart therapy are more likely to die in follow-up. It is tempting to use this fact to support restarting anticoagulants. However, when the causes of death are examined, the vast majority of deaths were unrelated to thrombosis or hemorrhage.² Patients with AF are older and have multiple comorbidities and life-limiting conditions. Accordingly, the results of this study are better used to engage patients in shared decision-making and contextualized in the broader picture of patients’ health and goals.⁵

Restarting anticoagulants after a GI hemorrhage is a difficult and high-stakes clinical decision. The study by Pappas et al. uses a simulation model to advance our understanding about the optimal timing to restart anticoagulants. By integrating the dynamic risk of ischemic stroke and recurrent hemorrhage following GI hemorrhage, they estimate the maximal benefit when anticoagulants are restarted between 30 days and 50 days after hemorrhage. The results of their analysis are best used to inform timing among patients where the decision to restart anticoagulants has already been made. The analysis also provides a useful starting point for shared decision-making by highlighting that the optimal net benefit is influenced by patient-to-patient variation in the underlying AF-related stroke risk and anticoagulant-associated rebleeding risk.

Disclosures: Dr. Shah has nothing to disclose. Dr. Eckman reports grants from Heart Rhythm Society/Boehringer-Ingelheim and grants from Bristol-Myers Squibb/Pfizer Education Consortium, outside the submitted work.

Anticoagulant use to prevent ischemic strokes in patients with atrial fibrillation (AF) continues to be one of the most challenging decisions facing patients and their physicians, in large part due to significant patient-to-patient variation in both AF-related stroke risk and anticoagulant-associated hemorrhage risk. Now, add a layer of complexity—.how should one approach anticoagulant use following an adverse event such as an acute upper gastrointestinal (GI) hemorrhage? On the one side, the risk of ischemic stroke, and on the other, the risk of recurrent bleeding, either of which can lead to death or disability. Making this decision requires humility, clinical acumen, shared decision-making, and data.

Data on this subject are sparse.^1,2 Observational studies show that patients who restart anticoagulants after GI hemorrhage experience fewer ischemic strokes. These studies also show that patients who restart anticoagulant therapy are healthier than those who do not—in measurable ways and, importantly, in unmeasurable ways. Thus far, observational studies have not sufficiently dealt with confounding by indication; that is, patients who restart anticoagulants are fundamentally different than patients who do not.

In this issue of the Journal of Hospital Medicine^®, Pappas et al. focus on the optimal timing of resuming oral anticoagulation in patients who have sustained acute upper GI bleeds while receiving oral anticoagulation for AF.³ They use a microsimulation modeling approach to address this question, by creating a synthetic population of patients reflective of age, gender, and comorbidities in a United States population of patients with AF. Using data from epidemiologic studies that describe the risk of rebleeding, hemorrhagic complications, and ischemic stroke as well as the quality of life associated with each of these events, the authors have constructed a decision analytic model to determine the optimal day to restart anticoagulation. This modeling approach mitigates confounding by indication, a limitation of observational studies. They report that the optimal day to restart anticoagulant therapy is in the range of 32-51 days. As one would predict, when using direct-acting anticoagulants and for patients with high stroke risk, the investigators find that restarting therapy earlier is associated with greater benefit. These findings help to untangle a knot of risk and benefits facing patients with AF following an acute GI hemorrhage.

Interpreting the results relies on an understanding of the strengths and weaknesses of simulation modeling and the data used in the analysis. Like any research method, the devil is in the details. Stitching together event rates and outcomes from multiple studies, the results of a simulation model are only as good as the studies the model draws from. In particular, assumptions regarding the time-dependent decline in rebleeding risk are a critical component of determining the optimal time to resume anticoagulation. The authors had to make multiple assumptions to project the 24-hour risk of rebleeding determined from the Rockall score to estimate the risk of rebleeding over the next days to months.⁴ Consequently, the results are likely overly precise. Practically, 30-50 days or four to eight weeks may better reflect the precision of the study findings.

Results on optimal timing of resuming anticoagulation therapy are most applicable for patients when the decision to restart anticoagulants has already been made. We part ways with the authors in their conclusion that these results confirm that anticoagulants should be restarted. There are multiple appropriate reasons why anticoagulant therapy should not be restarted following an acute upper GI hemorrhage. For example, in observational studies, patients not restarted on anticoagulant therapy were more likely to have a history of falls and to have had severe bleeds.¹ Furthermore, patients who do not restart therapy are more likely to die in follow-up. It is tempting to use this fact to support restarting anticoagulants. However, when the causes of death are examined, the vast majority of deaths were unrelated to thrombosis or hemorrhage.² Patients with AF are older and have multiple comorbidities and life-limiting conditions. Accordingly, the results of this study are better used to engage patients in shared decision-making and contextualized in the broader picture of patients’ health and goals.⁵

Restarting anticoagulants after a GI hemorrhage is a difficult and high-stakes clinical decision. The study by Pappas et al. uses a simulation model to advance our understanding about the optimal timing to restart anticoagulants. By integrating the dynamic risk of ischemic stroke and recurrent hemorrhage following GI hemorrhage, they estimate the maximal benefit when anticoagulants are restarted between 30 days and 50 days after hemorrhage. The results of their analysis are best used to inform timing among patients where the decision to restart anticoagulants has already been made. The analysis also provides a useful starting point for shared decision-making by highlighting that the optimal net benefit is influenced by patient-to-patient variation in the underlying AF-related stroke risk and anticoagulant-associated rebleeding risk.

Disclosures: Dr. Shah has nothing to disclose. Dr. Eckman reports grants from Heart Rhythm Society/Boehringer-Ingelheim and grants from Bristol-Myers Squibb/Pfizer Education Consortium, outside the submitted work.

Anticoagulant use to prevent ischemic strokes in patients with atrial fibrillation (AF) continues to be one of the most challenging decisions facing patients and their physicians, in large part due to significant patient-to-patient variation in both AF-related stroke risk and anticoagulant-associated hemorrhage risk. Now, add a layer of complexity—.how should one approach anticoagulant use following an adverse event such as an acute upper gastrointestinal (GI) hemorrhage? On the one side, the risk of ischemic stroke, and on the other, the risk of recurrent bleeding, either of which can lead to death or disability. Making this decision requires humility, clinical acumen, shared decision-making, and data.

Data on this subject are sparse.^1,2 Observational studies show that patients who restart anticoagulants after GI hemorrhage experience fewer ischemic strokes. These studies also show that patients who restart anticoagulant therapy are healthier than those who do not—in measurable ways and, importantly, in unmeasurable ways. Thus far, observational studies have not sufficiently dealt with confounding by indication; that is, patients who restart anticoagulants are fundamentally different than patients who do not.

In this issue of the Journal of Hospital Medicine^®, Pappas et al. focus on the optimal timing of resuming oral anticoagulation in patients who have sustained acute upper GI bleeds while receiving oral anticoagulation for AF.³ They use a microsimulation modeling approach to address this question, by creating a synthetic population of patients reflective of age, gender, and comorbidities in a United States population of patients with AF. Using data from epidemiologic studies that describe the risk of rebleeding, hemorrhagic complications, and ischemic stroke as well as the quality of life associated with each of these events, the authors have constructed a decision analytic model to determine the optimal day to restart anticoagulation. This modeling approach mitigates confounding by indication, a limitation of observational studies. They report that the optimal day to restart anticoagulant therapy is in the range of 32-51 days. As one would predict, when using direct-acting anticoagulants and for patients with high stroke risk, the investigators find that restarting therapy earlier is associated with greater benefit. These findings help to untangle a knot of risk and benefits facing patients with AF following an acute GI hemorrhage.

Interpreting the results relies on an understanding of the strengths and weaknesses of simulation modeling and the data used in the analysis. Like any research method, the devil is in the details. Stitching together event rates and outcomes from multiple studies, the results of a simulation model are only as good as the studies the model draws from. In particular, assumptions regarding the time-dependent decline in rebleeding risk are a critical component of determining the optimal time to resume anticoagulation. The authors had to make multiple assumptions to project the 24-hour risk of rebleeding determined from the Rockall score to estimate the risk of rebleeding over the next days to months.⁴ Consequently, the results are likely overly precise. Practically, 30-50 days or four to eight weeks may better reflect the precision of the study findings.

Results on optimal timing of resuming anticoagulation therapy are most applicable for patients when the decision to restart anticoagulants has already been made. We part ways with the authors in their conclusion that these results confirm that anticoagulants should be restarted. There are multiple appropriate reasons why anticoagulant therapy should not be restarted following an acute upper GI hemorrhage. For example, in observational studies, patients not restarted on anticoagulant therapy were more likely to have a history of falls and to have had severe bleeds.¹ Furthermore, patients who do not restart therapy are more likely to die in follow-up. It is tempting to use this fact to support restarting anticoagulants. However, when the causes of death are examined, the vast majority of deaths were unrelated to thrombosis or hemorrhage.² Patients with AF are older and have multiple comorbidities and life-limiting conditions. Accordingly, the results of this study are better used to engage patients in shared decision-making and contextualized in the broader picture of patients’ health and goals.⁵

Restarting anticoagulants after a GI hemorrhage is a difficult and high-stakes clinical decision. The study by Pappas et al. uses a simulation model to advance our understanding about the optimal timing to restart anticoagulants. By integrating the dynamic risk of ischemic stroke and recurrent hemorrhage following GI hemorrhage, they estimate the maximal benefit when anticoagulants are restarted between 30 days and 50 days after hemorrhage. The results of their analysis are best used to inform timing among patients where the decision to restart anticoagulants has already been made. The analysis also provides a useful starting point for shared decision-making by highlighting that the optimal net benefit is influenced by patient-to-patient variation in the underlying AF-related stroke risk and anticoagulant-associated rebleeding risk.

Disclosures: Dr. Shah has nothing to disclose. Dr. Eckman reports grants from Heart Rhythm Society/Boehringer-Ingelheim and grants from Bristol-Myers Squibb/Pfizer Education Consortium, outside the submitted work.

Women and racial and ethnic minorities remain underrepresented in senior faculty roles and academic leadership positions.¹ Participation in peer review and publication in medical journals are important components of academic advancement that are emphasized in the promotion process. These efforts offer recognition of expertise and increase visibility in the scientific community, which may enhance opportunities for networking and collaboration, and provide other opportunities for career advancement. In addition, abundant evidence shows that organizations benefit from diverse teams, with better quality decisions and increased productivity resulting from diverse ideas and perspectives.²

Numerous studies have highlighted the prevalence and persistence of disparities in peer review and authorship.^3,4 Much of this work has focused on gender though gaps in these measures likely exist for racial and ethnic minorities. Yet, there are few examples of journals implementing strategies to address disparities and track results of such efforts.⁵ While institutional barriers to advancement must be addressed, we believe that medical journals have an obligation to address unequal opportunities.

At the Journal of Hospital Medicine, we are committed to leading by example and developing approaches to create equity in all facets of journal leadership and authorship.⁶ The first step towards progress is to assess the current representation of women and racial and ethnic minorities in our journal community, including first and senior authors, invited expert contributors, reviewers, and editorial team members. Like most journals, we have not collected demographic information from authors or reviewers. But now, as part of the journal’s commitment to this cause, we request that everyone in the journal community (author, reviewer, editor) update their journal account (accessible at https://mc.manuscriptcentral.com/jhm) with demographic data, including gender, race, and ethnicity.

Inclusion of these data is voluntary. While each individual will be able to access and edit their personal demographic data, the individual data will remain private and unviewable to others. As such, it will not be available for nor will it be used in the manuscript review or decision process but rather for assessing our own inclusiveness. We will review these data in aggregate to broadly inform outreach efforts to promote diversity and inclusion in our author, invited expert contributor, reviewer, and journal leadership pools. We will report on the progress of these efforts in upcoming years.

We are committed to equity in providing opportunities for academic advancement across the journal community. Diversity and inclusion are important in raising the quality of the work that we publish. Different perspectives strengthen our journal and will help us continue to advance the field of Hospital Medicine.

Disclosures

The authors have nothing to disclose.

Women and racial and ethnic minorities remain underrepresented in senior faculty roles and academic leadership positions.¹ Participation in peer review and publication in medical journals are important components of academic advancement that are emphasized in the promotion process. These efforts offer recognition of expertise and increase visibility in the scientific community, which may enhance opportunities for networking and collaboration, and provide other opportunities for career advancement. In addition, abundant evidence shows that organizations benefit from diverse teams, with better quality decisions and increased productivity resulting from diverse ideas and perspectives.²

Numerous studies have highlighted the prevalence and persistence of disparities in peer review and authorship.^3,4 Much of this work has focused on gender though gaps in these measures likely exist for racial and ethnic minorities. Yet, there are few examples of journals implementing strategies to address disparities and track results of such efforts.⁵ While institutional barriers to advancement must be addressed, we believe that medical journals have an obligation to address unequal opportunities.

At the Journal of Hospital Medicine, we are committed to leading by example and developing approaches to create equity in all facets of journal leadership and authorship.⁶ The first step towards progress is to assess the current representation of women and racial and ethnic minorities in our journal community, including first and senior authors, invited expert contributors, reviewers, and editorial team members. Like most journals, we have not collected demographic information from authors or reviewers. But now, as part of the journal’s commitment to this cause, we request that everyone in the journal community (author, reviewer, editor) update their journal account (accessible at https://mc.manuscriptcentral.com/jhm) with demographic data, including gender, race, and ethnicity.

Inclusion of these data is voluntary. While each individual will be able to access and edit their personal demographic data, the individual data will remain private and unviewable to others. As such, it will not be available for nor will it be used in the manuscript review or decision process but rather for assessing our own inclusiveness. We will review these data in aggregate to broadly inform outreach efforts to promote diversity and inclusion in our author, invited expert contributor, reviewer, and journal leadership pools. We will report on the progress of these efforts in upcoming years.

We are committed to equity in providing opportunities for academic advancement across the journal community. Diversity and inclusion are important in raising the quality of the work that we publish. Different perspectives strengthen our journal and will help us continue to advance the field of Hospital Medicine.

Disclosures

The authors have nothing to disclose.

Women and racial and ethnic minorities remain underrepresented in senior faculty roles and academic leadership positions.¹ Participation in peer review and publication in medical journals are important components of academic advancement that are emphasized in the promotion process. These efforts offer recognition of expertise and increase visibility in the scientific community, which may enhance opportunities for networking and collaboration, and provide other opportunities for career advancement. In addition, abundant evidence shows that organizations benefit from diverse teams, with better quality decisions and increased productivity resulting from diverse ideas and perspectives.²

Numerous studies have highlighted the prevalence and persistence of disparities in peer review and authorship.^3,4 Much of this work has focused on gender though gaps in these measures likely exist for racial and ethnic minorities. Yet, there are few examples of journals implementing strategies to address disparities and track results of such efforts.⁵ While institutional barriers to advancement must be addressed, we believe that medical journals have an obligation to address unequal opportunities.

At the Journal of Hospital Medicine, we are committed to leading by example and developing approaches to create equity in all facets of journal leadership and authorship.⁶ The first step towards progress is to assess the current representation of women and racial and ethnic minorities in our journal community, including first and senior authors, invited expert contributors, reviewers, and editorial team members. Like most journals, we have not collected demographic information from authors or reviewers. But now, as part of the journal’s commitment to this cause, we request that everyone in the journal community (author, reviewer, editor) update their journal account (accessible at https://mc.manuscriptcentral.com/jhm) with demographic data, including gender, race, and ethnicity.

Inclusion of these data is voluntary. While each individual will be able to access and edit their personal demographic data, the individual data will remain private and unviewable to others. As such, it will not be available for nor will it be used in the manuscript review or decision process but rather for assessing our own inclusiveness. We will review these data in aggregate to broadly inform outreach efforts to promote diversity and inclusion in our author, invited expert contributor, reviewer, and journal leadership pools. We will report on the progress of these efforts in upcoming years.

We are committed to equity in providing opportunities for academic advancement across the journal community. Diversity and inclusion are important in raising the quality of the work that we publish. Different perspectives strengthen our journal and will help us continue to advance the field of Hospital Medicine.

Disclosures

The authors have nothing to disclose.

The United States (US) and Canada are the two highest per-capita consumers of opioids in the world;¹ both are struggling with unprecedented opioid-related mortality.^2,3 The nonmedical use of opioids is facilitated by diversion and defined as the transfer of drugs from lawful to unlawful channels of use^4,5 (eg, sharing legitimate prescriptions with family and friends⁶). Opioids and other controlled drugs are also diverted from healthcare facilities;^4,5,7,8 Canadian data suggest these incidents may be increasing (controlled-drug loss reports have doubled each year since 2015⁹).

The diversion of controlled drugs from hospitals affects patients, healthcare workers (HCWs), hospitals, and the public. Patients suffer insufficient analgesia or anesthesia, experience substandard care from impaired HCWs, and are at risk of infections from compromised syringes.^4,10,11 HCWs that divert are at risk of overdose and death; they also face regulatory censure, criminal prosecution, and civil malpractice suits.^12,13 Hospitals bear the cost of diverted drugs,^14,15 internal investigations,⁴ and follow-up care for affected patients,^4,13 and can be fined in excess of $4 million dollars for inadequate safeguards.¹⁶ Negative publicity highlights hospitals failing to self-regulate and report when diversion occurs, compromising public trust.^17-19 Finally, diverted drugs impact population health by contributing to drug misuse.

Hospitals face a critical problem: how does a hospital prevent the diversion of controlled drugs? Hospitals have not yet implemented safeguards needed to detect or understand how diversion occurs. For example, 79% of Canadian hospital controlled-drug loss reports are “unexplained losses,”⁹ demonstrating a lack of traceability needed to understand the root causes of the loss. A single US endoscopy clinic showed that $10,000 of propofol was unaccounted for over a four-week period.¹⁴ Although transactional discrepancies do not equate to diversion, they are a potential signal of diversion and highlight areas for improvement.¹⁵ The hospital medication-use process (MUP; eg, procurement, storage, preparation, prescription, dispensing, administration, waste, return, and removal) has multiple vulnerabilities that have been exploited. Published accounts of diversion include falsification of clinical documents, substitution of saline for medication, and theft.^4,20-23 Hospitals require guidance to assess their drug processes against known vulnerabilities and identify safeguards that may improve their capacity to prevent or detect diversion.

In this work, we provide a scoping review on the emerging topic of drug diversion to support hospitals. Scoping reviews can be a “preliminary attempt to provide an overview of existing literature that identifies areas where more research might be required.”²⁴ Past literature has identified sources of drugs for nonmedical use,^6,25,26 provided partial data on the quantities of stolen drug,^7,8 and estimated the rate of HCW diversion.^5,27-29 However, no reviews have focused on system gaps specific to hospital MUPs and diversion. Our review remedies this knowledge gap by consolidating known weaknesses and safeguards from peer- and nonpeer-reviewed articles. Drug diversion has been discussed at conferences and in news articles, case studies, and legal reports; excluding such discussion ignores substantive work that informs diversion practices in hospitals. Early indications suggest that hospitals have not yet implemented safeguards to properly identify when diversion has occurred, and consequently, lack the evidence to contribute to peer-reviewed literature. This article summarizes (1) clinical units, health professions, and stages of the MUP discussed, (2) contributors to diversion in hospitals, and (3) safeguards to prevent or detect diversion in hospitals.

Scoping Review

We followed Arksey and O’Malley’s six-step framework for scoping reviews,³⁰ with the exception of the optional consultation phase (step 6). We addressed three questions (step 1): what clinical units, health professions, or stages of the medication-use process are commonly discussed; what are the identified contributors to diversion in hospitals; and what safeguards have been described for prevention or detection of diversion in hospitals? We then identified relevant studies (step 2) by searching records published from January 2005 to June 2018 in MEDLINE, Embase, PsycINFO, CINAHL, Scopus, and Web of Science; the gray literature was also searched (see supplementary material for search terms).

All study designs were considered, including quantitative and qualitative methods, such as experiments, chart reviews and audit reports, surveys, focus groups, outbreak investigations, and literature reviews. Records were included (step 3) if abstracts met the Boolean logic criteria outlined in Appendix 1. If no abstract was available, then the full-text article was assessed. Prior to abstract screening, four reviewers (including R.R.) independently screened batches of 50 abstracts at a time to iteratively assess interrater reliability (IRR). Disagreements were resolved by consensus and the eligibility criteria were refined until IRR was achieved (Fleiss kappa > 0.65). Once IRR was achieved, the reviewers applied the criteria independently. For each eligible abstract, the full text was retrieved and assigned to a reviewer for independent assessment of eligibility. The abstract was reviewed if the full-text article was not available. Only articles published in English were included.

Reviewers charted findings from the full-text records (steps 4 and 5) by using themes defined a priori, specifically literature characteristics (eg, authors, year of publication), characteristics related to study method (eg, article type), variables related to our research questions (eg, variations by clinical unit, health profession), contributors to diversion, and safeguards to detect or prevent diversion. Inductive additions or modifications to the themes were proposed during the full-text review (eg, reviewers added a theme “name of drugs diverted” to identify drugs frequently reported as diverted) and accepted by consensus among the reviewers.

Scoping Review

The literature search generated 4,733 records of which 307 were duplicates and 4,009 were excluded on the basis of the eligibility criteria. The reviewers achieved 100% interrater agreement on the fourth round of abstract screening. Upon full-text review, 312 articles were included for data abstraction (Figure).

Literature Characteristics

Table 1 summarizes the characteristics of the included literature. The articles were published in a mix of peer-reviewed (137, 44%) and nonpeer-reviewed (175, 56%) sources. Some peer-reviewed articles did not use research methods, and some nonpeer-reviewed articles used research methods (eg, doctoral theses). Therefore, Table 1 categorizes the articles by research method (if applicable) and by peer-review status. The articles primarily originated in the United States (211, 68%) followed by Canada (79, 25%) and other countries (22, 7%). Most articles were commentaries, editorials, reports or news media, rather than formal studies presenting original data.

Literature Focus by Clinical Unit, Health Profession, and Stage of Medication-Use Process

Most articles did not focus the discussion on any one clinical unit, health profession, or stage of the MUP. Of the articles that made explicit mention of clinical units, hospital pharmacies and operating rooms were discussed most often, nurses were the most frequently highlighted health profession, and most stages of the MUP were discussed equally, with the exception of prescribing which was mentioned the least (Supplementary Table).

Contributors to Diversion

The literature describes a variety of contributors to drug diversion. Table 2 organizes these contributors by stage of the MUP and provides references for further discussion.

The diverse and system-wide contributors to diversion described in Table 2 support inappropriate access to controlled drugs and can delay the detection of diversion after it occurred. These contributors are more likely to occur in organizations that fail to adhere to drug-handling practices or to carefully review practices.^34,44

Diversion Safeguards in Hospitals

Table 3 summarizes published recommendations to mitigate the risk of diversion by stage of the MUP.

This review synthesizes a broad sample of peer- and nonpeer-reviewed literature to produce a consolidated list of known contributors (Table 2) and safeguards against (Table 3) controlled-drug diversion in hospitals. The literature describes an extensive list of ways drugs have been diverted in all stages of the MUP and can be exploited by all health professions in any clinical unit. Hospitals should be aware that nonclinical HCWs may also be at risk (eg, shipping and receiving personnel may handle drug shipments or returns, housekeeping may encounter partially filled vials in patient rooms). Patients and their families may also use some of the methods described in Table 2 (eg, acquiring fentanyl patches from unsecured waste receptacles and tampering with unsecured intravenous infusions).

Given the established presence of drug diversion in the literature,^5,7-9,96,97 hospitals should assess their clinical practices against these findings, review the associated references, and refer to existing guidance to better understand the intricacies of the topic.^{7,31,51,53,60,79} To accommodate variability in practice between hospitals, we suggest considering two underlying issues that recur in Tables 2 and 3 that will allow hospitals to systematically analyze their unique practices for each stage of the MUP.

Drug diversion in hospitals is a serious and urgent concern that requires immediate attention to mitigate harms. Past incidents of diversion have shown that hospitals have not yet implemented safeguards to fully account for drug losses, with resultant harms to patients, HCWs, hospitals themselves, and the general public. Further research is needed to identify system factors relevant to drug diversion, identify new safeguards, evaluate the effectiveness of known safeguards, and support adoption of best practices by hospitals and regulatory bodies.

Acknowledgments

The authors wish to thank Iveta Lewis and members of the HumanEra team (Carly Warren, Jessica Tomasi, Devika Jain, Maaike deVries, and Betty Chang) for screening and data extraction of the literature and to Peggy Robinson, Sylvia Hyland, and Sonia Pinkney for editing and commentary.

Disclosures

Ms. Reding and Ms. Hyland were employees of North York General Hospital at the time of this work. Dr. Hamilton and Ms. Tscheng are employees of ISMP Canada, a subcontractor to NYGH, during the conduct of the study. Mark Fan and Patricia Trbovich have received honoraria from BD Canada for presenting preliminary study findings at BD sponsored events.

Funding

This work was supported by Becton Dickinson (BD) Canada Inc. (grant #ROR2017-04260JH-NYGH). BD Canada had no involvement in study design; in the collection, analysis or interpretation of data; in the writing of the report; or in the decision to submit the article for publication.

The United States (US) and Canada are the two highest per-capita consumers of opioids in the world;¹ both are struggling with unprecedented opioid-related mortality.^2,3 The nonmedical use of opioids is facilitated by diversion and defined as the transfer of drugs from lawful to unlawful channels of use^4,5 (eg, sharing legitimate prescriptions with family and friends⁶). Opioids and other controlled drugs are also diverted from healthcare facilities;^4,5,7,8 Canadian data suggest these incidents may be increasing (controlled-drug loss reports have doubled each year since 2015⁹).

The diversion of controlled drugs from hospitals affects patients, healthcare workers (HCWs), hospitals, and the public. Patients suffer insufficient analgesia or anesthesia, experience substandard care from impaired HCWs, and are at risk of infections from compromised syringes.^4,10,11 HCWs that divert are at risk of overdose and death; they also face regulatory censure, criminal prosecution, and civil malpractice suits.^12,13 Hospitals bear the cost of diverted drugs,^14,15 internal investigations,⁴ and follow-up care for affected patients,^4,13 and can be fined in excess of $4 million dollars for inadequate safeguards.¹⁶ Negative publicity highlights hospitals failing to self-regulate and report when diversion occurs, compromising public trust.^17-19 Finally, diverted drugs impact population health by contributing to drug misuse.

Hospitals face a critical problem: how does a hospital prevent the diversion of controlled drugs? Hospitals have not yet implemented safeguards needed to detect or understand how diversion occurs. For example, 79% of Canadian hospital controlled-drug loss reports are “unexplained losses,”⁹ demonstrating a lack of traceability needed to understand the root causes of the loss. A single US endoscopy clinic showed that $10,000 of propofol was unaccounted for over a four-week period.¹⁴ Although transactional discrepancies do not equate to diversion, they are a potential signal of diversion and highlight areas for improvement.¹⁵ The hospital medication-use process (MUP; eg, procurement, storage, preparation, prescription, dispensing, administration, waste, return, and removal) has multiple vulnerabilities that have been exploited. Published accounts of diversion include falsification of clinical documents, substitution of saline for medication, and theft.^4,20-23 Hospitals require guidance to assess their drug processes against known vulnerabilities and identify safeguards that may improve their capacity to prevent or detect diversion.

In this work, we provide a scoping review on the emerging topic of drug diversion to support hospitals. Scoping reviews can be a “preliminary attempt to provide an overview of existing literature that identifies areas where more research might be required.”²⁴ Past literature has identified sources of drugs for nonmedical use,^6,25,26 provided partial data on the quantities of stolen drug,^7,8 and estimated the rate of HCW diversion.^5,27-29 However, no reviews have focused on system gaps specific to hospital MUPs and diversion. Our review remedies this knowledge gap by consolidating known weaknesses and safeguards from peer- and nonpeer-reviewed articles. Drug diversion has been discussed at conferences and in news articles, case studies, and legal reports; excluding such discussion ignores substantive work that informs diversion practices in hospitals. Early indications suggest that hospitals have not yet implemented safeguards to properly identify when diversion has occurred, and consequently, lack the evidence to contribute to peer-reviewed literature. This article summarizes (1) clinical units, health professions, and stages of the MUP discussed, (2) contributors to diversion in hospitals, and (3) safeguards to prevent or detect diversion in hospitals.

Scoping Review

We followed Arksey and O’Malley’s six-step framework for scoping reviews,³⁰ with the exception of the optional consultation phase (step 6). We addressed three questions (step 1): what clinical units, health professions, or stages of the medication-use process are commonly discussed; what are the identified contributors to diversion in hospitals; and what safeguards have been described for prevention or detection of diversion in hospitals? We then identified relevant studies (step 2) by searching records published from January 2005 to June 2018 in MEDLINE, Embase, PsycINFO, CINAHL, Scopus, and Web of Science; the gray literature was also searched (see supplementary material for search terms).

All study designs were considered, including quantitative and qualitative methods, such as experiments, chart reviews and audit reports, surveys, focus groups, outbreak investigations, and literature reviews. Records were included (step 3) if abstracts met the Boolean logic criteria outlined in Appendix 1. If no abstract was available, then the full-text article was assessed. Prior to abstract screening, four reviewers (including R.R.) independently screened batches of 50 abstracts at a time to iteratively assess interrater reliability (IRR). Disagreements were resolved by consensus and the eligibility criteria were refined until IRR was achieved (Fleiss kappa > 0.65). Once IRR was achieved, the reviewers applied the criteria independently. For each eligible abstract, the full text was retrieved and assigned to a reviewer for independent assessment of eligibility. The abstract was reviewed if the full-text article was not available. Only articles published in English were included.

Reviewers charted findings from the full-text records (steps 4 and 5) by using themes defined a priori, specifically literature characteristics (eg, authors, year of publication), characteristics related to study method (eg, article type), variables related to our research questions (eg, variations by clinical unit, health profession), contributors to diversion, and safeguards to detect or prevent diversion. Inductive additions or modifications to the themes were proposed during the full-text review (eg, reviewers added a theme “name of drugs diverted” to identify drugs frequently reported as diverted) and accepted by consensus among the reviewers.

Scoping Review

The literature search generated 4,733 records of which 307 were duplicates and 4,009 were excluded on the basis of the eligibility criteria. The reviewers achieved 100% interrater agreement on the fourth round of abstract screening. Upon full-text review, 312 articles were included for data abstraction (Figure).

Literature Characteristics

Table 1 summarizes the characteristics of the included literature. The articles were published in a mix of peer-reviewed (137, 44%) and nonpeer-reviewed (175, 56%) sources. Some peer-reviewed articles did not use research methods, and some nonpeer-reviewed articles used research methods (eg, doctoral theses). Therefore, Table 1 categorizes the articles by research method (if applicable) and by peer-review status. The articles primarily originated in the United States (211, 68%) followed by Canada (79, 25%) and other countries (22, 7%). Most articles were commentaries, editorials, reports or news media, rather than formal studies presenting original data.

Literature Focus by Clinical Unit, Health Profession, and Stage of Medication-Use Process

Most articles did not focus the discussion on any one clinical unit, health profession, or stage of the MUP. Of the articles that made explicit mention of clinical units, hospital pharmacies and operating rooms were discussed most often, nurses were the most frequently highlighted health profession, and most stages of the MUP were discussed equally, with the exception of prescribing which was mentioned the least (Supplementary Table).

Contributors to Diversion

The literature describes a variety of contributors to drug diversion. Table 2 organizes these contributors by stage of the MUP and provides references for further discussion.

The diverse and system-wide contributors to diversion described in Table 2 support inappropriate access to controlled drugs and can delay the detection of diversion after it occurred. These contributors are more likely to occur in organizations that fail to adhere to drug-handling practices or to carefully review practices.^34,44

Diversion Safeguards in Hospitals

Table 3 summarizes published recommendations to mitigate the risk of diversion by stage of the MUP.

This review synthesizes a broad sample of peer- and nonpeer-reviewed literature to produce a consolidated list of known contributors (Table 2) and safeguards against (Table 3) controlled-drug diversion in hospitals. The literature describes an extensive list of ways drugs have been diverted in all stages of the MUP and can be exploited by all health professions in any clinical unit. Hospitals should be aware that nonclinical HCWs may also be at risk (eg, shipping and receiving personnel may handle drug shipments or returns, housekeeping may encounter partially filled vials in patient rooms). Patients and their families may also use some of the methods described in Table 2 (eg, acquiring fentanyl patches from unsecured waste receptacles and tampering with unsecured intravenous infusions).

Given the established presence of drug diversion in the literature,^5,7-9,96,97 hospitals should assess their clinical practices against these findings, review the associated references, and refer to existing guidance to better understand the intricacies of the topic.^{7,31,51,53,60,79} To accommodate variability in practice between hospitals, we suggest considering two underlying issues that recur in Tables 2 and 3 that will allow hospitals to systematically analyze their unique practices for each stage of the MUP.

Drug diversion in hospitals is a serious and urgent concern that requires immediate attention to mitigate harms. Past incidents of diversion have shown that hospitals have not yet implemented safeguards to fully account for drug losses, with resultant harms to patients, HCWs, hospitals themselves, and the general public. Further research is needed to identify system factors relevant to drug diversion, identify new safeguards, evaluate the effectiveness of known safeguards, and support adoption of best practices by hospitals and regulatory bodies.

Acknowledgments

The authors wish to thank Iveta Lewis and members of the HumanEra team (Carly Warren, Jessica Tomasi, Devika Jain, Maaike deVries, and Betty Chang) for screening and data extraction of the literature and to Peggy Robinson, Sylvia Hyland, and Sonia Pinkney for editing and commentary.

Disclosures

Ms. Reding and Ms. Hyland were employees of North York General Hospital at the time of this work. Dr. Hamilton and Ms. Tscheng are employees of ISMP Canada, a subcontractor to NYGH, during the conduct of the study. Mark Fan and Patricia Trbovich have received honoraria from BD Canada for presenting preliminary study findings at BD sponsored events.

Funding

This work was supported by Becton Dickinson (BD) Canada Inc. (grant #ROR2017-04260JH-NYGH). BD Canada had no involvement in study design; in the collection, analysis or interpretation of data; in the writing of the report; or in the decision to submit the article for publication.

The United States (US) and Canada are the two highest per-capita consumers of opioids in the world;¹ both are struggling with unprecedented opioid-related mortality.^2,3 The nonmedical use of opioids is facilitated by diversion and defined as the transfer of drugs from lawful to unlawful channels of use^4,5 (eg, sharing legitimate prescriptions with family and friends⁶). Opioids and other controlled drugs are also diverted from healthcare facilities;^4,5,7,8 Canadian data suggest these incidents may be increasing (controlled-drug loss reports have doubled each year since 2015⁹).

The diversion of controlled drugs from hospitals affects patients, healthcare workers (HCWs), hospitals, and the public. Patients suffer insufficient analgesia or anesthesia, experience substandard care from impaired HCWs, and are at risk of infections from compromised syringes.^4,10,11 HCWs that divert are at risk of overdose and death; they also face regulatory censure, criminal prosecution, and civil malpractice suits.^12,13 Hospitals bear the cost of diverted drugs,^14,15 internal investigations,⁴ and follow-up care for affected patients,^4,13 and can be fined in excess of $4 million dollars for inadequate safeguards.¹⁶ Negative publicity highlights hospitals failing to self-regulate and report when diversion occurs, compromising public trust.^17-19 Finally, diverted drugs impact population health by contributing to drug misuse.

Hospitals face a critical problem: how does a hospital prevent the diversion of controlled drugs? Hospitals have not yet implemented safeguards needed to detect or understand how diversion occurs. For example, 79% of Canadian hospital controlled-drug loss reports are “unexplained losses,”⁹ demonstrating a lack of traceability needed to understand the root causes of the loss. A single US endoscopy clinic showed that $10,000 of propofol was unaccounted for over a four-week period.¹⁴ Although transactional discrepancies do not equate to diversion, they are a potential signal of diversion and highlight areas for improvement.¹⁵ The hospital medication-use process (MUP; eg, procurement, storage, preparation, prescription, dispensing, administration, waste, return, and removal) has multiple vulnerabilities that have been exploited. Published accounts of diversion include falsification of clinical documents, substitution of saline for medication, and theft.^4,20-23 Hospitals require guidance to assess their drug processes against known vulnerabilities and identify safeguards that may improve their capacity to prevent or detect diversion.

In this work, we provide a scoping review on the emerging topic of drug diversion to support hospitals. Scoping reviews can be a “preliminary attempt to provide an overview of existing literature that identifies areas where more research might be required.”²⁴ Past literature has identified sources of drugs for nonmedical use,^6,25,26 provided partial data on the quantities of stolen drug,^7,8 and estimated the rate of HCW diversion.^5,27-29 However, no reviews have focused on system gaps specific to hospital MUPs and diversion. Our review remedies this knowledge gap by consolidating known weaknesses and safeguards from peer- and nonpeer-reviewed articles. Drug diversion has been discussed at conferences and in news articles, case studies, and legal reports; excluding such discussion ignores substantive work that informs diversion practices in hospitals. Early indications suggest that hospitals have not yet implemented safeguards to properly identify when diversion has occurred, and consequently, lack the evidence to contribute to peer-reviewed literature. This article summarizes (1) clinical units, health professions, and stages of the MUP discussed, (2) contributors to diversion in hospitals, and (3) safeguards to prevent or detect diversion in hospitals.

Scoping Review

We followed Arksey and O’Malley’s six-step framework for scoping reviews,³⁰ with the exception of the optional consultation phase (step 6). We addressed three questions (step 1): what clinical units, health professions, or stages of the medication-use process are commonly discussed; what are the identified contributors to diversion in hospitals; and what safeguards have been described for prevention or detection of diversion in hospitals? We then identified relevant studies (step 2) by searching records published from January 2005 to June 2018 in MEDLINE, Embase, PsycINFO, CINAHL, Scopus, and Web of Science; the gray literature was also searched (see supplementary material for search terms).

All study designs were considered, including quantitative and qualitative methods, such as experiments, chart reviews and audit reports, surveys, focus groups, outbreak investigations, and literature reviews. Records were included (step 3) if abstracts met the Boolean logic criteria outlined in Appendix 1. If no abstract was available, then the full-text article was assessed. Prior to abstract screening, four reviewers (including R.R.) independently screened batches of 50 abstracts at a time to iteratively assess interrater reliability (IRR). Disagreements were resolved by consensus and the eligibility criteria were refined until IRR was achieved (Fleiss kappa > 0.65). Once IRR was achieved, the reviewers applied the criteria independently. For each eligible abstract, the full text was retrieved and assigned to a reviewer for independent assessment of eligibility. The abstract was reviewed if the full-text article was not available. Only articles published in English were included.

Reviewers charted findings from the full-text records (steps 4 and 5) by using themes defined a priori, specifically literature characteristics (eg, authors, year of publication), characteristics related to study method (eg, article type), variables related to our research questions (eg, variations by clinical unit, health profession), contributors to diversion, and safeguards to detect or prevent diversion. Inductive additions or modifications to the themes were proposed during the full-text review (eg, reviewers added a theme “name of drugs diverted” to identify drugs frequently reported as diverted) and accepted by consensus among the reviewers.

Scoping Review

The literature search generated 4,733 records of which 307 were duplicates and 4,009 were excluded on the basis of the eligibility criteria. The reviewers achieved 100% interrater agreement on the fourth round of abstract screening. Upon full-text review, 312 articles were included for data abstraction (Figure).

Literature Characteristics

Table 1 summarizes the characteristics of the included literature. The articles were published in a mix of peer-reviewed (137, 44%) and nonpeer-reviewed (175, 56%) sources. Some peer-reviewed articles did not use research methods, and some nonpeer-reviewed articles used research methods (eg, doctoral theses). Therefore, Table 1 categorizes the articles by research method (if applicable) and by peer-review status. The articles primarily originated in the United States (211, 68%) followed by Canada (79, 25%) and other countries (22, 7%). Most articles were commentaries, editorials, reports or news media, rather than formal studies presenting original data.

Literature Focus by Clinical Unit, Health Profession, and Stage of Medication-Use Process

Most articles did not focus the discussion on any one clinical unit, health profession, or stage of the MUP. Of the articles that made explicit mention of clinical units, hospital pharmacies and operating rooms were discussed most often, nurses were the most frequently highlighted health profession, and most stages of the MUP were discussed equally, with the exception of prescribing which was mentioned the least (Supplementary Table).

Contributors to Diversion

The literature describes a variety of contributors to drug diversion. Table 2 organizes these contributors by stage of the MUP and provides references for further discussion.

The diverse and system-wide contributors to diversion described in Table 2 support inappropriate access to controlled drugs and can delay the detection of diversion after it occurred. These contributors are more likely to occur in organizations that fail to adhere to drug-handling practices or to carefully review practices.^34,44

Diversion Safeguards in Hospitals

Table 3 summarizes published recommendations to mitigate the risk of diversion by stage of the MUP.

This review synthesizes a broad sample of peer- and nonpeer-reviewed literature to produce a consolidated list of known contributors (Table 2) and safeguards against (Table 3) controlled-drug diversion in hospitals. The literature describes an extensive list of ways drugs have been diverted in all stages of the MUP and can be exploited by all health professions in any clinical unit. Hospitals should be aware that nonclinical HCWs may also be at risk (eg, shipping and receiving personnel may handle drug shipments or returns, housekeeping may encounter partially filled vials in patient rooms). Patients and their families may also use some of the methods described in Table 2 (eg, acquiring fentanyl patches from unsecured waste receptacles and tampering with unsecured intravenous infusions).

Given the established presence of drug diversion in the literature,^5,7-9,96,97 hospitals should assess their clinical practices against these findings, review the associated references, and refer to existing guidance to better understand the intricacies of the topic.^{7,31,51,53,60,79} To accommodate variability in practice between hospitals, we suggest considering two underlying issues that recur in Tables 2 and 3 that will allow hospitals to systematically analyze their unique practices for each stage of the MUP.

Drug diversion in hospitals is a serious and urgent concern that requires immediate attention to mitigate harms. Past incidents of diversion have shown that hospitals have not yet implemented safeguards to fully account for drug losses, with resultant harms to patients, HCWs, hospitals themselves, and the general public. Further research is needed to identify system factors relevant to drug diversion, identify new safeguards, evaluate the effectiveness of known safeguards, and support adoption of best practices by hospitals and regulatory bodies.

Acknowledgments

The authors wish to thank Iveta Lewis and members of the HumanEra team (Carly Warren, Jessica Tomasi, Devika Jain, Maaike deVries, and Betty Chang) for screening and data extraction of the literature and to Peggy Robinson, Sylvia Hyland, and Sonia Pinkney for editing and commentary.

Disclosures

Ms. Reding and Ms. Hyland were employees of North York General Hospital at the time of this work. Dr. Hamilton and Ms. Tscheng are employees of ISMP Canada, a subcontractor to NYGH, during the conduct of the study. Mark Fan and Patricia Trbovich have received honoraria from BD Canada for presenting preliminary study findings at BD sponsored events.

Funding

This work was supported by Becton Dickinson (BD) Canada Inc. (grant #ROR2017-04260JH-NYGH). BD Canada had no involvement in study design; in the collection, analysis or interpretation of data; in the writing of the report; or in the decision to submit the article for publication.

Research evidence synthesis involves the aggregation of available information using well-defined and transparent methods to search, summarize, and interpret a body of literature, frequently following a systematic review approach. A scoping review is a relatively new approach to evidence synthesis and differs from systematic reviews in its purpose and aims.¹ The purpose of a scoping review is to provide an overview of the available research evidence without producing a summary answer to a discrete research question.² Scoping reviews can be useful for answering broad questions, such as “What information has been presented on this topic in the literature?” and for gathering and assessing information prior to conducting a systematic review.¹

In this issue of the Journal of Hospital Medicine, Fan et al. used a scoping review to identify information available in the literature on contributors to loss and theft of controlled drugs in hospitals and the safeguards that have been suggested to address these diversions.³ The authors followed Arksey and O’Malley’s framework for scoping reviews and the PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews) checklist in reporting findings.^2,4

Scoping reviews describe existing literature and other sources of information commonly include findings from a range of different study designs and methods.⁵ The broad scope of the collected information makes using formal meta-analytic methods difficult, if not impossible. Results of a scoping review often focus on the range of content identified, and quantitative assessment is often limited to a tally of the number of sources reporting a particular issue or recommendation. In contrast, systematic reviews commonly select the information sources by requiring specific study types, such as randomized controlled trials, and imposing quality standards, such as adequate allocation concealment, and place their emphasis on synthesizing data to address a specific research question. (Table) By focusing on specific studies, the synthesis component in a systematic review often takes the form of a meta-analysis in which the results of multiple scientific studies are combined to develop a summary conclusion, such as a common effect estimate, along with an evaluation of its heterogeneity across studies.

A scoping review can be a particularly useful approach when the information on a topic has not been comprehensively reviewed or is complex and diverse.⁶ Munn et al. proposed several objectives that can be achieved utilizing the scoping review framework, including identifying types of existing evidence in a given field, clarifying key concepts or definitions in the literature, surveying how research is conducted on a certain topic, identifying key characteristics related to a certain topic, and identifying knowledge gaps.¹ When choosing to use a scoping review approach, it is important that the objective of the review align with the review’s indication or purpose.

Scoping reviews, like systematic reviews, require comprehensive and structured searches of the literature to maximize the capture of relevant information, provide reproducible results, and decrease potential bias from flawed implementations. The methodological framework for scoping reviews was developed by Arksey and O’Malley¹ and further refined by Levac et al.⁷ and the Joanna Briggs Institute.^6,8 Arksey and O’Malley’s framework for scoping reviews consists of the following six steps:

• Step 1: Identify the research question—the research question should be clearly defined and usually broad in scope to provide extensive coverage.
• Step 2: Identify relevant studies—the search strategy should be thorough and broad in scope and typically include electronic databases, reference lists, hand searches, and gray literature (ie, substantive or scholarly information that has not been formally published and often is not peer-reviewed), including conference abstracts, presentations, regulatory data, working papers, and patents.
• Step 3: Study selection—the study selection process can include post hoc, or modified, inclusion and exclusion criteria as new ideas emerge during the process of gathering and reviewing information.
• Step 4: Chart the data—the data extraction process in a scoping review is called data charting and involves the use of a data charting form to extract the relevant information from the reviewed literature.
• Step 5: Collate, summarize, and report the results—the description of the scope of the literature is commonly presented in tables and charts according to key themes.
• Optional Step 6: Consultation exercise—in this optional step, stakeholders outside the study review team are invited to provide their insights to inform and validate findings from the scoping review.

Since the number of studies included in a scoping review can be substantial, several study team members may participate in the review process. When multiple reviewers are employed, the team ought to conduct a calibration exercise at each step of the review process to ensure adequate interrater agreement. In addition, the PRISMA-ScR guidelines should be followed when reporting findings from scoping reviews to facilitate complete, transparent, and consistent reporting in the literature.⁴

The scoping review approach has several limitations. Scoping reviews do not formally evaluate the quality of evidence and often gather information from a wide range of study designs and methods. By design, the number of studies included in the review process can be sizable. Thus, a large study team is typically needed to screen the large number of studies and other sources for potential inclusion in the scoping review. Because scoping reviews provide a descriptive account of available information, this often leads to broad, less defined searches that require multiple structured strategies focused on alternative sets of themes. Hand searching the literature is therefore necessary to ensure the validity of this process. Scoping reviews do not provide a synthesized result or answer to a specific question, but rather provide an overview of the available literature. Even though statements regarding the quality of evidence and formal synthesis are avoided, the scoping review approach is not necessarily easier or faster than the systematic review approach. Scoping reviews require a substantial amount of time to complete due to the wide coverage of the search implicit in the approach.

Like other studies, scoping reviews are at risk for bias from different sources. Critical appraisal of the risk of bias in scoping reviews is not considered mandatory, but some scoping reviews may include a bias assessment. Even if bias is not formally assessed, that does not mean that bias does not exist. For example, selection bias may occur if the scoping review does not identify all available data on a topic and the resulting descriptive account of available information is flawed.

Fan et al. used the scoping review approach to examine the available information on contributors to and safeguards against controlled-drug losses and theft (drug diversion) in the hospital setting.³ The authors addressed the following questions: (1) “What clinical units, health professions, or stages of the medication-use process are commonly discussed?” (2) “What are the identified contributors to diversion in hospitals?” and (3) “What safeguards to prevent or detect diversion in hospitals have been described?” Part of the rationale for using a scoping review approach was to permit the inclusion of a wide range of sources falling outside the typical peer-reviewed article. The authors comment that the stigmatized topic of drug diversion frequently falls outside the peer-reviewed literature and emphasize the importance of including such sources as conferences, news articles, and legal reports. The search strategy included electronic research databases, such as Web of Science, as well as an extensive gray literature search. Multiple reviewers were included in the process and a calibration exercise was conducted to ensure consistency in the selection of articles and to improve interrater agreement. The scoping review identified contributors to controlled-drug diversion and suggested safeguards to address them in the hospital setting.

Methodological approaches to evidence synthesis vary, and new methods continue to emerge to meet different research objectives, including evidence mapping,⁹ concept analysis,¹⁰ rapid reviews,¹¹ and others.¹² Choosing the right approach may not be straightforward. Researchers may need to seek guidance from methodologists, including epidemiologists, statisticians, and information specialists, when choosing an appropriate review approach to ensure that the review methods are suitable for the objectives of the review.

Disclosures

The authors have no conflicts of interest to disclose.

Financial Disclosures

The authors have no financial relationships relevant to this article to disclose.

Research evidence synthesis involves the aggregation of available information using well-defined and transparent methods to search, summarize, and interpret a body of literature, frequently following a systematic review approach. A scoping review is a relatively new approach to evidence synthesis and differs from systematic reviews in its purpose and aims.¹ The purpose of a scoping review is to provide an overview of the available research evidence without producing a summary answer to a discrete research question.² Scoping reviews can be useful for answering broad questions, such as “What information has been presented on this topic in the literature?” and for gathering and assessing information prior to conducting a systematic review.¹

In this issue of the Journal of Hospital Medicine, Fan et al. used a scoping review to identify information available in the literature on contributors to loss and theft of controlled drugs in hospitals and the safeguards that have been suggested to address these diversions.³ The authors followed Arksey and O’Malley’s framework for scoping reviews and the PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews) checklist in reporting findings.^2,4

Scoping reviews describe existing literature and other sources of information commonly include findings from a range of different study designs and methods.⁵ The broad scope of the collected information makes using formal meta-analytic methods difficult, if not impossible. Results of a scoping review often focus on the range of content identified, and quantitative assessment is often limited to a tally of the number of sources reporting a particular issue or recommendation. In contrast, systematic reviews commonly select the information sources by requiring specific study types, such as randomized controlled trials, and imposing quality standards, such as adequate allocation concealment, and place their emphasis on synthesizing data to address a specific research question. (Table) By focusing on specific studies, the synthesis component in a systematic review often takes the form of a meta-analysis in which the results of multiple scientific studies are combined to develop a summary conclusion, such as a common effect estimate, along with an evaluation of its heterogeneity across studies.

A scoping review can be a particularly useful approach when the information on a topic has not been comprehensively reviewed or is complex and diverse.⁶ Munn et al. proposed several objectives that can be achieved utilizing the scoping review framework, including identifying types of existing evidence in a given field, clarifying key concepts or definitions in the literature, surveying how research is conducted on a certain topic, identifying key characteristics related to a certain topic, and identifying knowledge gaps.¹ When choosing to use a scoping review approach, it is important that the objective of the review align with the review’s indication or purpose.

Scoping reviews, like systematic reviews, require comprehensive and structured searches of the literature to maximize the capture of relevant information, provide reproducible results, and decrease potential bias from flawed implementations. The methodological framework for scoping reviews was developed by Arksey and O’Malley¹ and further refined by Levac et al.⁷ and the Joanna Briggs Institute.^6,8 Arksey and O’Malley’s framework for scoping reviews consists of the following six steps:

• Step 1: Identify the research question—the research question should be clearly defined and usually broad in scope to provide extensive coverage.
• Step 2: Identify relevant studies—the search strategy should be thorough and broad in scope and typically include electronic databases, reference lists, hand searches, and gray literature (ie, substantive or scholarly information that has not been formally published and often is not peer-reviewed), including conference abstracts, presentations, regulatory data, working papers, and patents.
• Step 3: Study selection—the study selection process can include post hoc, or modified, inclusion and exclusion criteria as new ideas emerge during the process of gathering and reviewing information.
• Step 4: Chart the data—the data extraction process in a scoping review is called data charting and involves the use of a data charting form to extract the relevant information from the reviewed literature.
• Step 5: Collate, summarize, and report the results—the description of the scope of the literature is commonly presented in tables and charts according to key themes.
• Optional Step 6: Consultation exercise—in this optional step, stakeholders outside the study review team are invited to provide their insights to inform and validate findings from the scoping review.

Since the number of studies included in a scoping review can be substantial, several study team members may participate in the review process. When multiple reviewers are employed, the team ought to conduct a calibration exercise at each step of the review process to ensure adequate interrater agreement. In addition, the PRISMA-ScR guidelines should be followed when reporting findings from scoping reviews to facilitate complete, transparent, and consistent reporting in the literature.⁴

The scoping review approach has several limitations. Scoping reviews do not formally evaluate the quality of evidence and often gather information from a wide range of study designs and methods. By design, the number of studies included in the review process can be sizable. Thus, a large study team is typically needed to screen the large number of studies and other sources for potential inclusion in the scoping review. Because scoping reviews provide a descriptive account of available information, this often leads to broad, less defined searches that require multiple structured strategies focused on alternative sets of themes. Hand searching the literature is therefore necessary to ensure the validity of this process. Scoping reviews do not provide a synthesized result or answer to a specific question, but rather provide an overview of the available literature. Even though statements regarding the quality of evidence and formal synthesis are avoided, the scoping review approach is not necessarily easier or faster than the systematic review approach. Scoping reviews require a substantial amount of time to complete due to the wide coverage of the search implicit in the approach.

Like other studies, scoping reviews are at risk for bias from different sources. Critical appraisal of the risk of bias in scoping reviews is not considered mandatory, but some scoping reviews may include a bias assessment. Even if bias is not formally assessed, that does not mean that bias does not exist. For example, selection bias may occur if the scoping review does not identify all available data on a topic and the resulting descriptive account of available information is flawed.

Fan et al. used the scoping review approach to examine the available information on contributors to and safeguards against controlled-drug losses and theft (drug diversion) in the hospital setting.³ The authors addressed the following questions: (1) “What clinical units, health professions, or stages of the medication-use process are commonly discussed?” (2) “What are the identified contributors to diversion in hospitals?” and (3) “What safeguards to prevent or detect diversion in hospitals have been described?” Part of the rationale for using a scoping review approach was to permit the inclusion of a wide range of sources falling outside the typical peer-reviewed article. The authors comment that the stigmatized topic of drug diversion frequently falls outside the peer-reviewed literature and emphasize the importance of including such sources as conferences, news articles, and legal reports. The search strategy included electronic research databases, such as Web of Science, as well as an extensive gray literature search. Multiple reviewers were included in the process and a calibration exercise was conducted to ensure consistency in the selection of articles and to improve interrater agreement. The scoping review identified contributors to controlled-drug diversion and suggested safeguards to address them in the hospital setting.

Methodological approaches to evidence synthesis vary, and new methods continue to emerge to meet different research objectives, including evidence mapping,⁹ concept analysis,¹⁰ rapid reviews,¹¹ and others.¹² Choosing the right approach may not be straightforward. Researchers may need to seek guidance from methodologists, including epidemiologists, statisticians, and information specialists, when choosing an appropriate review approach to ensure that the review methods are suitable for the objectives of the review.

Disclosures

The authors have no conflicts of interest to disclose.

Financial Disclosures

The authors have no financial relationships relevant to this article to disclose.

Research evidence synthesis involves the aggregation of available information using well-defined and transparent methods to search, summarize, and interpret a body of literature, frequently following a systematic review approach. A scoping review is a relatively new approach to evidence synthesis and differs from systematic reviews in its purpose and aims.¹ The purpose of a scoping review is to provide an overview of the available research evidence without producing a summary answer to a discrete research question.² Scoping reviews can be useful for answering broad questions, such as “What information has been presented on this topic in the literature?” and for gathering and assessing information prior to conducting a systematic review.¹

In this issue of the Journal of Hospital Medicine, Fan et al. used a scoping review to identify information available in the literature on contributors to loss and theft of controlled drugs in hospitals and the safeguards that have been suggested to address these diversions.³ The authors followed Arksey and O’Malley’s framework for scoping reviews and the PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews) checklist in reporting findings.^2,4

Scoping reviews describe existing literature and other sources of information commonly include findings from a range of different study designs and methods.⁵ The broad scope of the collected information makes using formal meta-analytic methods difficult, if not impossible. Results of a scoping review often focus on the range of content identified, and quantitative assessment is often limited to a tally of the number of sources reporting a particular issue or recommendation. In contrast, systematic reviews commonly select the information sources by requiring specific study types, such as randomized controlled trials, and imposing quality standards, such as adequate allocation concealment, and place their emphasis on synthesizing data to address a specific research question. (Table) By focusing on specific studies, the synthesis component in a systematic review often takes the form of a meta-analysis in which the results of multiple scientific studies are combined to develop a summary conclusion, such as a common effect estimate, along with an evaluation of its heterogeneity across studies.

A scoping review can be a particularly useful approach when the information on a topic has not been comprehensively reviewed or is complex and diverse.⁶ Munn et al. proposed several objectives that can be achieved utilizing the scoping review framework, including identifying types of existing evidence in a given field, clarifying key concepts or definitions in the literature, surveying how research is conducted on a certain topic, identifying key characteristics related to a certain topic, and identifying knowledge gaps.¹ When choosing to use a scoping review approach, it is important that the objective of the review align with the review’s indication or purpose.

Scoping reviews, like systematic reviews, require comprehensive and structured searches of the literature to maximize the capture of relevant information, provide reproducible results, and decrease potential bias from flawed implementations. The methodological framework for scoping reviews was developed by Arksey and O’Malley¹ and further refined by Levac et al.⁷ and the Joanna Briggs Institute.^6,8 Arksey and O’Malley’s framework for scoping reviews consists of the following six steps:

• Step 1: Identify the research question—the research question should be clearly defined and usually broad in scope to provide extensive coverage.
• Step 2: Identify relevant studies—the search strategy should be thorough and broad in scope and typically include electronic databases, reference lists, hand searches, and gray literature (ie, substantive or scholarly information that has not been formally published and often is not peer-reviewed), including conference abstracts, presentations, regulatory data, working papers, and patents.
• Step 3: Study selection—the study selection process can include post hoc, or modified, inclusion and exclusion criteria as new ideas emerge during the process of gathering and reviewing information.
• Step 4: Chart the data—the data extraction process in a scoping review is called data charting and involves the use of a data charting form to extract the relevant information from the reviewed literature.
• Step 5: Collate, summarize, and report the results—the description of the scope of the literature is commonly presented in tables and charts according to key themes.
• Optional Step 6: Consultation exercise—in this optional step, stakeholders outside the study review team are invited to provide their insights to inform and validate findings from the scoping review.

Since the number of studies included in a scoping review can be substantial, several study team members may participate in the review process. When multiple reviewers are employed, the team ought to conduct a calibration exercise at each step of the review process to ensure adequate interrater agreement. In addition, the PRISMA-ScR guidelines should be followed when reporting findings from scoping reviews to facilitate complete, transparent, and consistent reporting in the literature.⁴

The scoping review approach has several limitations. Scoping reviews do not formally evaluate the quality of evidence and often gather information from a wide range of study designs and methods. By design, the number of studies included in the review process can be sizable. Thus, a large study team is typically needed to screen the large number of studies and other sources for potential inclusion in the scoping review. Because scoping reviews provide a descriptive account of available information, this often leads to broad, less defined searches that require multiple structured strategies focused on alternative sets of themes. Hand searching the literature is therefore necessary to ensure the validity of this process. Scoping reviews do not provide a synthesized result or answer to a specific question, but rather provide an overview of the available literature. Even though statements regarding the quality of evidence and formal synthesis are avoided, the scoping review approach is not necessarily easier or faster than the systematic review approach. Scoping reviews require a substantial amount of time to complete due to the wide coverage of the search implicit in the approach.

Like other studies, scoping reviews are at risk for bias from different sources. Critical appraisal of the risk of bias in scoping reviews is not considered mandatory, but some scoping reviews may include a bias assessment. Even if bias is not formally assessed, that does not mean that bias does not exist. For example, selection bias may occur if the scoping review does not identify all available data on a topic and the resulting descriptive account of available information is flawed.

Fan et al. used the scoping review approach to examine the available information on contributors to and safeguards against controlled-drug losses and theft (drug diversion) in the hospital setting.³ The authors addressed the following questions: (1) “What clinical units, health professions, or stages of the medication-use process are commonly discussed?” (2) “What are the identified contributors to diversion in hospitals?” and (3) “What safeguards to prevent or detect diversion in hospitals have been described?” Part of the rationale for using a scoping review approach was to permit the inclusion of a wide range of sources falling outside the typical peer-reviewed article. The authors comment that the stigmatized topic of drug diversion frequently falls outside the peer-reviewed literature and emphasize the importance of including such sources as conferences, news articles, and legal reports. The search strategy included electronic research databases, such as Web of Science, as well as an extensive gray literature search. Multiple reviewers were included in the process and a calibration exercise was conducted to ensure consistency in the selection of articles and to improve interrater agreement. The scoping review identified contributors to controlled-drug diversion and suggested safeguards to address them in the hospital setting.

Methodological approaches to evidence synthesis vary, and new methods continue to emerge to meet different research objectives, including evidence mapping,⁹ concept analysis,¹⁰ rapid reviews,¹¹ and others.¹² Choosing the right approach may not be straightforward. Researchers may need to seek guidance from methodologists, including epidemiologists, statisticians, and information specialists, when choosing an appropriate review approach to ensure that the review methods are suitable for the objectives of the review.

Disclosures

The authors have no conflicts of interest to disclose.

Financial Disclosures

The authors have no financial relationships relevant to this article to disclose.

The patient safety field rightly focuses on identifying and addressing problems with systems of care. From the patient’s perspective, however, underlying systems issues might be less critical than another unspoken question: can I trust the people who are taking care of me? Last year, a popular podcast¹ detailed the shocking story of Dallas neurosurgeon Christopher Duntsch, who was responsible for the death of two patients and severe injuries in dozens of other patients over two years. Although fellow surgeons had raised concerns about his surgical skill and professionalism almost immediately after he entered practice, multiple hospitals allowed him to continue operating until the Texas Medical Board revoked his license. Duntsch was ultimately prosecuted, convicted, and sentenced to life imprisonment, in what is believed to be the first case of a physician receiving criminal punishment for malpractice.

Only a small proportion of clinicians repeatedly harm patients as Duntsch did, and the harm they cause accounts for only a small share of the preventable adverse events that patients experience. Understandably, cases of individual clinicians who directly harm patients tend to capture the public’s attention, as they vividly illustrate how vulnerable patients are when they entrust their health to a clinician. As a result, these cases have a significant effect on the patient’s trust in healthcare institutions.

In this issue of the Journal of Hospital Medicine, Fan and colleagues² describe the problem of controlled-substance diversion in hospitals and review the contributors and potential solutions to this issue. Their thorough and insightful review highlights a growing problem that is probably invisible to most hospitalists. Diversion of controlled substances can happen at any stage of the medication use process, from procurement to disposal and drugs can be diverted by healthcare workers, nonclinical staff, patients, and caregivers. Perhaps most concerning to hospitalists, diversion at the prescribing and administration stages can directly affect patient care. Strategies used to individualize pain control, such as using flexible dose ranges for opioids, can be manipulated to facilitate diversion at the expense of the patient’s suffering.

The review presents a comprehensive summary of safeguards against diversion at each stage of the medication use process and appropriately emphasizes system-level solutions. These include analyzing electronic health record data to identify unusual patterns of controlled substance use and developing dedicated diversion investigation teams. These measures, if implemented, are likely to be effective at reducing the risk of diversion. However, given the complexity of medication use, eliminating this risk is unrealistic. Opioids are used in more than half of all nonsurgical hospital admissions;³ although this proportion may be decreasing due to efforts to curb opioid overprescribing, many hospitalized patients still require opioids or other controlled substances for symptom control. The opportunity to divert controlled substances will always be present.

Eliminating the problem of drug diversion in hospitals will require addressing the individuals who divert controlled substances and strengthening the medication safety system. The term “impaired clinician” is used to describe clinicians who cannot provide competent care due to illness, mental health, or a substance-use disorder. In an influential 2006 commentary,Leape and Fromson made the case that physician performance impairment is often a symptom of underlying disorders, ranging from short-term, reversible issues (eg, an episode of burnout or depression) to long-term problems that can lead to permanent consequences (ie, physical illness or substance-use disorders).⁴ In this framework, a clinician who diverts controlled substances represents a particularly extreme example of the broader problem of physicians who are unable to perform their professional responsibilities.

Leape and Fromson called for proactively identifying clinicians at risk of performance failure and intervening to remediate or discipline them before patients are harmed. To accomplish this, they envisioned a system with three key characteristics:

• Fairness: All physicians should be subject to regular assessment, and the same standards should be applied to all physicians in the same discipline.
• Objectivity: Performance assessment should be based on objective data.
• Responsiveness: Physicians with performance issues should be identified and given feedback promptly, and provided with opportunities for remediation and assistance when underlying conditions are affecting their performance.

Some progress has been made toward this goal, especially in identifying underlying factors that predispose to performance problems.⁵ There is also greater awareness of underlying factors that may predispose to more subtle performance deterioration. The recent focus on burnout and well-being among physicians is long overdue, and the recent Charter on Physician Well-Being⁶ articulates important principles for healthcare organizations to address this epidemic. Substance-use disorder is a recognized risk factor for performance impairment. Physicians have a higher rate of prescription drug abuse and a similar overall rate of substance-use disorders compared to the general population. While there is limited research around the risk factors for drug diversion by physicians, qualitative studies⁷ of physicians undergoing treatment for substance-use disorders found that most began diverting drugs to manage physical pain, emotional or psychiatric distress, or acutely stressful situations. It is plausible that many burned out or depressed clinicians are turning to illicit substances to self-medicate increasing the risk of diversion.

However, 13 years after Leape and Fromson’s commentary was published, it is difficult to conclude that their vision has been achieved. Objectivity in physician performance assessment is still lacking, and most practicing physicians do not receive any form of regular assessment. This places the onus on members of the healthcare team to identify poorly performing colleagues before patients are harmed. Although nearly all states mandate that physicians report impaired colleagues to either the state medical board or a physician rehabilitation program, healthcare professionals are often reluctant⁸ to report colleagues with performance issues, and clinicians are also unlikely⁹ to self-report mental health or substance-use issues due to stigma and fear that their ability to practice may be at risk.

Even when colleagues do raise alarms—as was the case with Dr. Duntsch, who required treatment for a substance-use disorder during residency—existing regulatory mechanisms either lack evidence of effectiveness or are not applied consistently. State licensing boards play a crucial role in identifying problems with clinicians and have the power to authorize remediation or disciplinary measures. However, individual states vary widely¹⁰ in their likelihood of disciplining physicians for similar offenses. The board certification process is intended to ensure that only fully competent physicians can practice medicine independently. However, there is little evidence that the certification process ensures that clinicians maintain their skills, and significant controversy has accompanied efforts to revise the maintenance of certification process. The medical malpractice system aims to improve patient safety by ensuring compensation when patients are injured and by deterring substandard clinicians from practicing. Unfortunately, the system often fails to meet this goal, as malpractice claims are rarely filed even when patients are harmed due to negligent care.¹¹

Given the widespread availability of controlled substances in hospitals, comprehensive solutions must incorporate the systems-based solutions proffered by Fan and colleagues and address individual clinicians (and staff) who divert drugs. These clinicians are likely to share some of the same risk factors as clinicians who cannot perform their professional responsibilities for other reasons. Major system changes are necessary to minimize the risk of short-term conditions that could affect physician performance (such as burnout) and develop robust methods to identify clinicians with longer-term issues affecting their performance (such as substance-use disorders).

Although individual clinician performance problems likely account for a small proportion of adverse events, these issues strike at the heart of the physician-patient relationship and have a profound impact on patients’ trust in the healthcare system. Healthcare organizations must maintain transparent and effective processes for addressing performance failures such as drug diversion by clinicians, even if these processes are rarely deployed.

Disclosures

The author does not have any conflict of interest to report.

The patient safety field rightly focuses on identifying and addressing problems with systems of care. From the patient’s perspective, however, underlying systems issues might be less critical than another unspoken question: can I trust the people who are taking care of me? Last year, a popular podcast¹ detailed the shocking story of Dallas neurosurgeon Christopher Duntsch, who was responsible for the death of two patients and severe injuries in dozens of other patients over two years. Although fellow surgeons had raised concerns about his surgical skill and professionalism almost immediately after he entered practice, multiple hospitals allowed him to continue operating until the Texas Medical Board revoked his license. Duntsch was ultimately prosecuted, convicted, and sentenced to life imprisonment, in what is believed to be the first case of a physician receiving criminal punishment for malpractice.

Only a small proportion of clinicians repeatedly harm patients as Duntsch did, and the harm they cause accounts for only a small share of the preventable adverse events that patients experience. Understandably, cases of individual clinicians who directly harm patients tend to capture the public’s attention, as they vividly illustrate how vulnerable patients are when they entrust their health to a clinician. As a result, these cases have a significant effect on the patient’s trust in healthcare institutions.

In this issue of the Journal of Hospital Medicine, Fan and colleagues² describe the problem of controlled-substance diversion in hospitals and review the contributors and potential solutions to this issue. Their thorough and insightful review highlights a growing problem that is probably invisible to most hospitalists. Diversion of controlled substances can happen at any stage of the medication use process, from procurement to disposal and drugs can be diverted by healthcare workers, nonclinical staff, patients, and caregivers. Perhaps most concerning to hospitalists, diversion at the prescribing and administration stages can directly affect patient care. Strategies used to individualize pain control, such as using flexible dose ranges for opioids, can be manipulated to facilitate diversion at the expense of the patient’s suffering.

The review presents a comprehensive summary of safeguards against diversion at each stage of the medication use process and appropriately emphasizes system-level solutions. These include analyzing electronic health record data to identify unusual patterns of controlled substance use and developing dedicated diversion investigation teams. These measures, if implemented, are likely to be effective at reducing the risk of diversion. However, given the complexity of medication use, eliminating this risk is unrealistic. Opioids are used in more than half of all nonsurgical hospital admissions;³ although this proportion may be decreasing due to efforts to curb opioid overprescribing, many hospitalized patients still require opioids or other controlled substances for symptom control. The opportunity to divert controlled substances will always be present.

Eliminating the problem of drug diversion in hospitals will require addressing the individuals who divert controlled substances and strengthening the medication safety system. The term “impaired clinician” is used to describe clinicians who cannot provide competent care due to illness, mental health, or a substance-use disorder. In an influential 2006 commentary,Leape and Fromson made the case that physician performance impairment is often a symptom of underlying disorders, ranging from short-term, reversible issues (eg, an episode of burnout or depression) to long-term problems that can lead to permanent consequences (ie, physical illness or substance-use disorders).⁴ In this framework, a clinician who diverts controlled substances represents a particularly extreme example of the broader problem of physicians who are unable to perform their professional responsibilities.

Leape and Fromson called for proactively identifying clinicians at risk of performance failure and intervening to remediate or discipline them before patients are harmed. To accomplish this, they envisioned a system with three key characteristics:

• Fairness: All physicians should be subject to regular assessment, and the same standards should be applied to all physicians in the same discipline.
• Objectivity: Performance assessment should be based on objective data.
• Responsiveness: Physicians with performance issues should be identified and given feedback promptly, and provided with opportunities for remediation and assistance when underlying conditions are affecting their performance.

Some progress has been made toward this goal, especially in identifying underlying factors that predispose to performance problems.⁵ There is also greater awareness of underlying factors that may predispose to more subtle performance deterioration. The recent focus on burnout and well-being among physicians is long overdue, and the recent Charter on Physician Well-Being⁶ articulates important principles for healthcare organizations to address this epidemic. Substance-use disorder is a recognized risk factor for performance impairment. Physicians have a higher rate of prescription drug abuse and a similar overall rate of substance-use disorders compared to the general population. While there is limited research around the risk factors for drug diversion by physicians, qualitative studies⁷ of physicians undergoing treatment for substance-use disorders found that most began diverting drugs to manage physical pain, emotional or psychiatric distress, or acutely stressful situations. It is plausible that many burned out or depressed clinicians are turning to illicit substances to self-medicate increasing the risk of diversion.

However, 13 years after Leape and Fromson’s commentary was published, it is difficult to conclude that their vision has been achieved. Objectivity in physician performance assessment is still lacking, and most practicing physicians do not receive any form of regular assessment. This places the onus on members of the healthcare team to identify poorly performing colleagues before patients are harmed. Although nearly all states mandate that physicians report impaired colleagues to either the state medical board or a physician rehabilitation program, healthcare professionals are often reluctant⁸ to report colleagues with performance issues, and clinicians are also unlikely⁹ to self-report mental health or substance-use issues due to stigma and fear that their ability to practice may be at risk.

Even when colleagues do raise alarms—as was the case with Dr. Duntsch, who required treatment for a substance-use disorder during residency—existing regulatory mechanisms either lack evidence of effectiveness or are not applied consistently. State licensing boards play a crucial role in identifying problems with clinicians and have the power to authorize remediation or disciplinary measures. However, individual states vary widely¹⁰ in their likelihood of disciplining physicians for similar offenses. The board certification process is intended to ensure that only fully competent physicians can practice medicine independently. However, there is little evidence that the certification process ensures that clinicians maintain their skills, and significant controversy has accompanied efforts to revise the maintenance of certification process. The medical malpractice system aims to improve patient safety by ensuring compensation when patients are injured and by deterring substandard clinicians from practicing. Unfortunately, the system often fails to meet this goal, as malpractice claims are rarely filed even when patients are harmed due to negligent care.¹¹

Given the widespread availability of controlled substances in hospitals, comprehensive solutions must incorporate the systems-based solutions proffered by Fan and colleagues and address individual clinicians (and staff) who divert drugs. These clinicians are likely to share some of the same risk factors as clinicians who cannot perform their professional responsibilities for other reasons. Major system changes are necessary to minimize the risk of short-term conditions that could affect physician performance (such as burnout) and develop robust methods to identify clinicians with longer-term issues affecting their performance (such as substance-use disorders).

Although individual clinician performance problems likely account for a small proportion of adverse events, these issues strike at the heart of the physician-patient relationship and have a profound impact on patients’ trust in the healthcare system. Healthcare organizations must maintain transparent and effective processes for addressing performance failures such as drug diversion by clinicians, even if these processes are rarely deployed.

Disclosures

The author does not have any conflict of interest to report.

The patient safety field rightly focuses on identifying and addressing problems with systems of care. From the patient’s perspective, however, underlying systems issues might be less critical than another unspoken question: can I trust the people who are taking care of me? Last year, a popular podcast¹ detailed the shocking story of Dallas neurosurgeon Christopher Duntsch, who was responsible for the death of two patients and severe injuries in dozens of other patients over two years. Although fellow surgeons had raised concerns about his surgical skill and professionalism almost immediately after he entered practice, multiple hospitals allowed him to continue operating until the Texas Medical Board revoked his license. Duntsch was ultimately prosecuted, convicted, and sentenced to life imprisonment, in what is believed to be the first case of a physician receiving criminal punishment for malpractice.

Only a small proportion of clinicians repeatedly harm patients as Duntsch did, and the harm they cause accounts for only a small share of the preventable adverse events that patients experience. Understandably, cases of individual clinicians who directly harm patients tend to capture the public’s attention, as they vividly illustrate how vulnerable patients are when they entrust their health to a clinician. As a result, these cases have a significant effect on the patient’s trust in healthcare institutions.

In this issue of the Journal of Hospital Medicine, Fan and colleagues² describe the problem of controlled-substance diversion in hospitals and review the contributors and potential solutions to this issue. Their thorough and insightful review highlights a growing problem that is probably invisible to most hospitalists. Diversion of controlled substances can happen at any stage of the medication use process, from procurement to disposal and drugs can be diverted by healthcare workers, nonclinical staff, patients, and caregivers. Perhaps most concerning to hospitalists, diversion at the prescribing and administration stages can directly affect patient care. Strategies used to individualize pain control, such as using flexible dose ranges for opioids, can be manipulated to facilitate diversion at the expense of the patient’s suffering.

The review presents a comprehensive summary of safeguards against diversion at each stage of the medication use process and appropriately emphasizes system-level solutions. These include analyzing electronic health record data to identify unusual patterns of controlled substance use and developing dedicated diversion investigation teams. These measures, if implemented, are likely to be effective at reducing the risk of diversion. However, given the complexity of medication use, eliminating this risk is unrealistic. Opioids are used in more than half of all nonsurgical hospital admissions;³ although this proportion may be decreasing due to efforts to curb opioid overprescribing, many hospitalized patients still require opioids or other controlled substances for symptom control. The opportunity to divert controlled substances will always be present.

Eliminating the problem of drug diversion in hospitals will require addressing the individuals who divert controlled substances and strengthening the medication safety system. The term “impaired clinician” is used to describe clinicians who cannot provide competent care due to illness, mental health, or a substance-use disorder. In an influential 2006 commentary,Leape and Fromson made the case that physician performance impairment is often a symptom of underlying disorders, ranging from short-term, reversible issues (eg, an episode of burnout or depression) to long-term problems that can lead to permanent consequences (ie, physical illness or substance-use disorders).⁴ In this framework, a clinician who diverts controlled substances represents a particularly extreme example of the broader problem of physicians who are unable to perform their professional responsibilities.

Leape and Fromson called for proactively identifying clinicians at risk of performance failure and intervening to remediate or discipline them before patients are harmed. To accomplish this, they envisioned a system with three key characteristics:

• Fairness: All physicians should be subject to regular assessment, and the same standards should be applied to all physicians in the same discipline.
• Objectivity: Performance assessment should be based on objective data.
• Responsiveness: Physicians with performance issues should be identified and given feedback promptly, and provided with opportunities for remediation and assistance when underlying conditions are affecting their performance.

Some progress has been made toward this goal, especially in identifying underlying factors that predispose to performance problems.⁵ There is also greater awareness of underlying factors that may predispose to more subtle performance deterioration. The recent focus on burnout and well-being among physicians is long overdue, and the recent Charter on Physician Well-Being⁶ articulates important principles for healthcare organizations to address this epidemic. Substance-use disorder is a recognized risk factor for performance impairment. Physicians have a higher rate of prescription drug abuse and a similar overall rate of substance-use disorders compared to the general population. While there is limited research around the risk factors for drug diversion by physicians, qualitative studies⁷ of physicians undergoing treatment for substance-use disorders found that most began diverting drugs to manage physical pain, emotional or psychiatric distress, or acutely stressful situations. It is plausible that many burned out or depressed clinicians are turning to illicit substances to self-medicate increasing the risk of diversion.

However, 13 years after Leape and Fromson’s commentary was published, it is difficult to conclude that their vision has been achieved. Objectivity in physician performance assessment is still lacking, and most practicing physicians do not receive any form of regular assessment. This places the onus on members of the healthcare team to identify poorly performing colleagues before patients are harmed. Although nearly all states mandate that physicians report impaired colleagues to either the state medical board or a physician rehabilitation program, healthcare professionals are often reluctant⁸ to report colleagues with performance issues, and clinicians are also unlikely⁹ to self-report mental health or substance-use issues due to stigma and fear that their ability to practice may be at risk.

Even when colleagues do raise alarms—as was the case with Dr. Duntsch, who required treatment for a substance-use disorder during residency—existing regulatory mechanisms either lack evidence of effectiveness or are not applied consistently. State licensing boards play a crucial role in identifying problems with clinicians and have the power to authorize remediation or disciplinary measures. However, individual states vary widely¹⁰ in their likelihood of disciplining physicians for similar offenses. The board certification process is intended to ensure that only fully competent physicians can practice medicine independently. However, there is little evidence that the certification process ensures that clinicians maintain their skills, and significant controversy has accompanied efforts to revise the maintenance of certification process. The medical malpractice system aims to improve patient safety by ensuring compensation when patients are injured and by deterring substandard clinicians from practicing. Unfortunately, the system often fails to meet this goal, as malpractice claims are rarely filed even when patients are harmed due to negligent care.¹¹

Given the widespread availability of controlled substances in hospitals, comprehensive solutions must incorporate the systems-based solutions proffered by Fan and colleagues and address individual clinicians (and staff) who divert drugs. These clinicians are likely to share some of the same risk factors as clinicians who cannot perform their professional responsibilities for other reasons. Major system changes are necessary to minimize the risk of short-term conditions that could affect physician performance (such as burnout) and develop robust methods to identify clinicians with longer-term issues affecting their performance (such as substance-use disorders).

Although individual clinician performance problems likely account for a small proportion of adverse events, these issues strike at the heart of the physician-patient relationship and have a profound impact on patients’ trust in the healthcare system. Healthcare organizations must maintain transparent and effective processes for addressing performance failures such as drug diversion by clinicians, even if these processes are rarely deployed.

Disclosures

The author does not have any conflict of interest to report.

The field of pediatric hospital medicine has seen tremendous growth in scholarship in the past decade. To obtain a wide view of advancements in the field from the current literature, the authors selected 18 English-language journals (Table 1) across four domains believed to be relevant to the practice of pediatric hospital medicine, including hospital medicine, pediatrics, emergency care, and medical education. The median Hirsch index (h-index) of the selected journals was 131. A goal of 10, a number that could maximally benefit consumers of the finished product, was set as the final number of articles to be selected.

Guiding principles for the initial selection included novelty of hypotheses, study design, significance of results, and likelihood to change pediatric hospital medicine practice from both the community and academic hospital perspectives. Journals were assigned randomly to each author for review and assignments were switched after six months to limit potential bias in coverage. A three-stage review process was employed. The authors initially independently reviewed titles and abstracts from 13,296 articles published between January 2018 and December 2018 and rated them according to their likelihood to be included in the final set of 10 articles and their broad applicability to pediatric hospital medicine. This resulted in 99 studies that were selected for further review. Next, the authors were assigned a subset of the 99 articles for further review; each author rated the articles independently based on their likelihood of inclusion in the final 10-article set. At this stage, 75 articles were excluded. Finally, all remaining 24 articles were reviewed independently and in depth by both authors.

Ten articles were selected by consensus formation, and the authors presented their findings at the 2019 Society for Hospital Medicine annual meeting. From these 10 articles, six were determined to be most impactful to current practice; these articles are presented below. After discussing the study background, an overview, key results, limitations of the study, important findings (Table 2), and implications for practice and future research are presented.

Interventions to Reduce Over-Utilized Tests and Treatments in Bronchiolitis. Tyler A, et al. Pediatrics. 2018;141(6):e20170485. ¹

Background

The American Academy of Pediatrics (AAP) published clinical practice guidelines (CPG) for bronchiolitis in 2014.² However, unnecessary tests and interventions continue to be ordered and used on children with bronchiolitis that are not recommended by the guidelines. In this quality improvement project, the authors sought to increase compliance with the AAP CPG for bronchiolitis by reducing chest x-rays (CXR) to <20%, respiratory viral testing (RVT) to <15%, and use of bronchodilators to <20%.

Study Overview and Results

This project took place at a free-standing children’s hospital and included urgent care locations. Authors obtained pre-intervention data through two bronchiolitis seasons in 2013 and 2014 for patients aged 1-23 months with a primary or secondary diagnosis of bronchiolitis and who did not require admission to the Intensive Care Unit (ICU). The intervention period was from December 2015 to April 2016. All sites simultaneously implemented their interventions, which included education of care team members and families, updated order sets, and electronic health record (EHR)-generated e-mails that provided data looking at peer ranking statistics for each intervention, CXR, RVT, and bronchodilator usage. A data dashboard was created to display real-time utilization of the studied interventions. Providers were also asked to sign a pledge that they would reduce unnecessary testing and treatment. As balancing measures, the numbers of patients presenting to the Emergency Room (ER) or readmitted within seven days of an ED visit or admission for bronchiolitis were tracked; patients who required ICU levels of care during their first admission or on readmission were also tracked. Statistically significant decreases in CXR ordering from 39.5% to 27.2%, RVT ordering from 31.9% to 26.3%, and any bronchodilator usage from 34.2% to 21.5% were noted. No difference pre- and postintervention in patients readmitted to the ICU was found, and length of stay (LOS) between groups was not statistically significant.

Limitations

As all interventions were initiated simultaneously, identifying which individual or subset of interventions was responsible for changing provider behavior was impossible. More patients postintervention were admitted under observation status and under a milder All Patient Refined Diagnosis Related Groups (APR DRGs) severity index, which may indicate a less-sick cohort of patients in this group. Since the LOS and number of patients readmitted to the ICU were similar in both groups, it is unlikely that the postintervention group represented a less-sick cohort.

Important Findings and Implications

This QI project highlighted novel ways to implement and emphasize the importance of compliance to CPG. A provider pledge may be helpful in reinforcing to all providers the idea that the institution is committed to guideline implementation. Comparing individual provider data and having a real-time dashboard with group performance can help reinforce goals and progress toward them at the group, site, and individual patient population levels.

Development and Validation of a Calculator for Estimating the Probability of Urinary Tract Infection in Young Febrile Children. Shaikh N, et al. JAMA Pediatrics. 2018;172(6):550-556.³

Background

The prevalence of urinary tract infections (UTIs) in children under 2 years of age that present to the emergency department (ED) with fever is about 7%.⁴ After clinical examination, providers obtaining a urinalysis must then determine if empirical antibiotics are warranted for a suspected UTI. This study describes the development of a novel calculator, UTICalc that estimates the pretest probability of a UTI based on clinical findings and the posttest probability of a UTI based on laboratory results.

Study Overview and Results

This study features a single center, nested, case-control design that looked retrospectively at 542 children aged 2-24 months who presented to the ED from January 2007 to April 2013 with fever and had a catheterized urinalysis obtained. Patients were then matched with randomly selected children without a UTI to create a training database. Five models using different variables were developed, including one with only clinical characteristics and four that combined clinical characteristics with differing laboratory values. The area under the curve of the “clinical model” was 0.80, while those of the remaining four models ranged from 0.97 to 0.98. The clinical model showed a sensitivity of 95% and specificity of 35% in the training database, while the four other models showed sensitivities ranging from 93% to 96% and specificities ranging from 91% to 93%. The models were then validated using a cohort of children aged 2-24 months who presented to the ED with fever from July 2015 to December 2016; the UTI prevalence in this cohort was 7.8%. Finally, using a hypothetical cohort of 1,000 children being evaluated for a UTI, the authors showed that UTICalc reduced the numbers of urine samples obtained by 8.1% and missed UTIs from 3 to 0 compared with following AAP guidelines.⁵

Limitations

The training database was created retrospectively at a single institution and is subject to local practice patterns. The proposed calculator creates an algorithm that is meant to be used in a setting where the pretest probability for a UTI is reasonably high based on criteria from the AAP UTI guidelines.

Important Findings and Implications

UTICalc could be a great tool for providers to guide testing for UTIs in children aged 2-24 months presenting with a fever. Given further study at multiple sites and settings, including outpatient clinics, UTICalc could have significant implications for reducing unnecessary testing and treatment in febrile children.

Lost Earnings and Nonmedical Expenses of Pediatric Hospitalizations. Chang LV, et al. Pediatrics. 2018;142(3):e20180195.⁶

Background

Although medical expenses related to hospitalization can be significant for many families, nonmedical costs, such as transportation, parking, meals, and lost earnings from missed days at work, are also important to consider. These hardships can lead to challenges in postdischarge follow-up and adherence to discharge instructions, both of which lead to hospital readmissions. This article presents a cross-sectional analysis at a large, free-standing children’s hospital that participated in the Hospital-to-Home Outcomes Study (H2O). The authors sought to determine whether families with more financial or social hardships are affected disproportionately by nonmedical costs related to hospitalizations.

Study Overview and Results

A total of 1,372 children were included and children with lengths of stay >13 days were excluded. Face-to-face parental surveys were conducted and included questions on parental education, employment status, sick leave flexibility, and measures of financial and social hardship. The study authors calculated a total cost burden (TCB) based on nonmedical costs estimated at the time of the survey, including lost wages and expenses during the hospitalization. A daily cost burden (DCB) based on length of hospital stay and daily cost burden as a percentage of daily income (DCBi) were also calculated. The median TCB was $112.80, and the median DCB was $51.40. The median DCBi showed that the median household had 45% of their daily income depleted by nonmedical expenses related to their hospitalization. Those who reported more financial or social hardships had a higher median DCBi; if ≥3 financial hardships were reported, 86% of the daily household income was depleted.

Limitations

The study was conducted at a single institution with a number of existing support systems in place to help unburden families of hospitalized children. Non-English-speaking families were excluded. A face-to-face survey may have influenced parental responses regarding social and financial hardships.

Important Findings and Implications

Nonmedical costs of hospitalized children can be quantified and disproportionately affect those experiencing financial and social hardships. Hospitalists should be aware of these findings and find ways within their hospital systems to provide support for families both during and after hospitalizations.

A Prescription for Note Bloat: An Effective Progress Note Template. Kahn D, et al. Journal of Hospital Medicine. 2018;13(6):378-382.⁷

Background

Although electronic health records (EHRs) have improved the speed and legibility of documentation, the harm of “note bloat,” defined as multiple pages of nonessential information which leaves key aspects buried or lost, is prevalent. In this prospective, quality improvement study across four internal medicine residency programs, the authors investigated a bundled intervention consisting of didactic teaching and an electronic progress note template on note quality, length, and timeliness.

Study overview and results

Notes pre- and postintervention were graded using a tool that considered the general impression of the note, its score on the validated Physician Documentation Quality Instrument (PDQI-9),⁸ and a questionnaire based on the Accreditation Council for Graduate Medical Education competency note checklist.⁹ Analyzing 200 preintervention and 199 postintervention notes, significant improvement was seen in general impression scores, all PDQI-9 domains, and 6 of 13 note competency questionnaire items. The mean number of lines in the note decreased by 25%, and the mean completion time when the note was signed was 1 hour and 15 minutes earlier. The greatest impact on shortening notes involved a reduction in the auto-population of laboratory and imaging studies.

Limitations

The study was unblinded. The authors attempted to minimize bias with an objective questionnaire and employed multiple graders per note; however, poor interrater reliability was obtained. Postintervention, 70% of all residents used the template. At one of the four institutions, evidence of note quality improvement despite low template use was found. At another institution, no improvement in note quality was reported despite relatively high template uptake. Local culture and institutional buy-in may be factors affecting these results. In addition, pre- and postintervention notes were examined in the same academic year; thus, the effects seen may be due, in part, to resident maturation. Generalizability to nonacademic institutions and the durability of the intervention are additional concerns.

Important Findings and Implications

Resident education on documentation and an EHR progress note template incorporating best practices can effectively combat “note bloat” and lead to higher quality and shorter notes that are completed earlier in the day. This solution has significant implications for improving transitions of care, handoffs, and patient safety.

Time to Pathogen Detection for Non-Ill Versus Ill-Appearing Infants ≤60 Days Old with Bacteremia and Meningitis. Aronson PL, et al. Hospital Pediatrics 2018;8 (7):379-384.¹⁰

Background

The routine evaluation of febrile infants aged ≤60 days old often involves blood and cerebrospinal (CSF) fluid evaluations, and many infants are hospitalized while waiting for culture results. A previous study of febrile infants showed that 91% of the pathogenic organisms could be identified on blood culture within 24 hours and that 96% could be identified within 36 hours; 81% of the bacterial pathogens present were detected on CSF culture within 36 hours.¹¹

Study Overview and Results

In this large, multicenter study of infants presenting to the Emergency Departments (EDs) of 10 children’s hospitals over a five-year study period, the authors investigated the time to pathogen detection in blood and CSF for infants aged ≤60 days with bacteremia and/or bacterial meningitis; whether the time to detection differed for non-ill and ill infants was also examined. Ill- versus non-ill-appearance was determined by a medical record review of the physical exam looking for one of 13 key words (eg, “ill-appearing,” “toxic,” “lethargic,” etc.). A total of 381 infants were included. Overall, 88% of the pathogens present were detected in blood culture within 24 hours and 95% were detected within 36 hours. In CSF, 89% of the pathogens present were detected within 24 hours, and 95% were detected within 36 hours. In infants with bacteremia who were non-ill-appearing, 85% of the blood pathogens were detected within 24 hours.

Limitations

The median time to detection for blood culture pathogens for ill-appearing versus non-ill-appearing infants was shorter by just one hour, but 15% of the non-ill infants had a positive blood culture after 24 hours. However, the prevalence of bacteremia and meningitis in non-ill-appearing infants is likely low; the authors did not report the total number of febrile infants evaluated by EDs in the study.

Important Findings and Implications

Most positive blood and/or CSF cultures for infants aged ≤60 days will yield results by 24 hours; 95% of the pathogens present could be detected within 36 hours. Sending a non-ill-appearing febrile infant home at 24 hours may miss 15% of the instances of bacteremia, but the overall low prevalence of invasive bacterial infection in infants should be considered.

The High-Value Care Rounding Tool: Development and Validity Evidence. McDaniel CE, et al. Academic Medicine. 2018;93(2):199-206.¹²

Background

Providing high-value care (HVC) to patients is a struggle for physicians and healthcare systems. Although physicians teaching trainees HVC practices could be an effective way to increase cost-conscious care, the best practices for teaching HVC remain unknown. To fill this gap, the authors developed a tool to measure the frequency and content of observable HVC teaching and evaluated the validity of the tool within a pediatric inpatient setting.

Study Overview and Results

The HVC rounding tool was developed through several phases from conception to validation. The research group used a modified Delphi method to construct the tool using a consensus building process based on opinions from content experts in the field of HVC, from a variety of specialties, experience levels, and geographic areas of the United States. Each item of the HVC instrument was rated by these experts, and, from their evaluations and surveys, an 11-item HVC tool was constructed. A pilot of the tool was performed to establish internal validity and interrater reliability based on observations of 148 patient encounters. From this process, a final 10-item HVC rounding tool emerged, including domains in quality, cost, and patient values. A few items included giving positive feedback for not doing an unnecessary test, discussing whether a patient needs to stay inpatient or meets discharge criteria, and customizing a care plan to align with family values and goals. The final iteration of the tool had no rater disagreements within the quality and patient values domain and only one disagreement within the cost domain.

Limitations

This tool was validated at a single pediatric institution, and, thus, the generalizability of the tool has not been established. The authors note that the Delphi panelists used for the construction of the tool were from a medical subspecialty background and not surgical backgrounds, which limits its applicability from a surgical perspective. The tool does not allow for differentiation between lengthy discussions or brief comments presented during rounds.

Important Findings and Implications

The HVC rounding tool is both innovative and timely. Pediatric hospitalists are leaders in family-centered care, and this tool allows assessment of whether important concepts of high-value care are discussed at the bedside. A multisite educational study using this tool would be welcome.

Disclosures

The authors have no conflicts of interest to declare.

The field of pediatric hospital medicine has seen tremendous growth in scholarship in the past decade. To obtain a wide view of advancements in the field from the current literature, the authors selected 18 English-language journals (Table 1) across four domains believed to be relevant to the practice of pediatric hospital medicine, including hospital medicine, pediatrics, emergency care, and medical education. The median Hirsch index (h-index) of the selected journals was 131. A goal of 10, a number that could maximally benefit consumers of the finished product, was set as the final number of articles to be selected.

Guiding principles for the initial selection included novelty of hypotheses, study design, significance of results, and likelihood to change pediatric hospital medicine practice from both the community and academic hospital perspectives. Journals were assigned randomly to each author for review and assignments were switched after six months to limit potential bias in coverage. A three-stage review process was employed. The authors initially independently reviewed titles and abstracts from 13,296 articles published between January 2018 and December 2018 and rated them according to their likelihood to be included in the final set of 10 articles and their broad applicability to pediatric hospital medicine. This resulted in 99 studies that were selected for further review. Next, the authors were assigned a subset of the 99 articles for further review; each author rated the articles independently based on their likelihood of inclusion in the final 10-article set. At this stage, 75 articles were excluded. Finally, all remaining 24 articles were reviewed independently and in depth by both authors.

Ten articles were selected by consensus formation, and the authors presented their findings at the 2019 Society for Hospital Medicine annual meeting. From these 10 articles, six were determined to be most impactful to current practice; these articles are presented below. After discussing the study background, an overview, key results, limitations of the study, important findings (Table 2), and implications for practice and future research are presented.

Interventions to Reduce Over-Utilized Tests and Treatments in Bronchiolitis. Tyler A, et al. Pediatrics. 2018;141(6):e20170485. ¹

Background

The American Academy of Pediatrics (AAP) published clinical practice guidelines (CPG) for bronchiolitis in 2014.² However, unnecessary tests and interventions continue to be ordered and used on children with bronchiolitis that are not recommended by the guidelines. In this quality improvement project, the authors sought to increase compliance with the AAP CPG for bronchiolitis by reducing chest x-rays (CXR) to <20%, respiratory viral testing (RVT) to <15%, and use of bronchodilators to <20%.

Study Overview and Results

This project took place at a free-standing children’s hospital and included urgent care locations. Authors obtained pre-intervention data through two bronchiolitis seasons in 2013 and 2014 for patients aged 1-23 months with a primary or secondary diagnosis of bronchiolitis and who did not require admission to the Intensive Care Unit (ICU). The intervention period was from December 2015 to April 2016. All sites simultaneously implemented their interventions, which included education of care team members and families, updated order sets, and electronic health record (EHR)-generated e-mails that provided data looking at peer ranking statistics for each intervention, CXR, RVT, and bronchodilator usage. A data dashboard was created to display real-time utilization of the studied interventions. Providers were also asked to sign a pledge that they would reduce unnecessary testing and treatment. As balancing measures, the numbers of patients presenting to the Emergency Room (ER) or readmitted within seven days of an ED visit or admission for bronchiolitis were tracked; patients who required ICU levels of care during their first admission or on readmission were also tracked. Statistically significant decreases in CXR ordering from 39.5% to 27.2%, RVT ordering from 31.9% to 26.3%, and any bronchodilator usage from 34.2% to 21.5% were noted. No difference pre- and postintervention in patients readmitted to the ICU was found, and length of stay (LOS) between groups was not statistically significant.

Limitations

As all interventions were initiated simultaneously, identifying which individual or subset of interventions was responsible for changing provider behavior was impossible. More patients postintervention were admitted under observation status and under a milder All Patient Refined Diagnosis Related Groups (APR DRGs) severity index, which may indicate a less-sick cohort of patients in this group. Since the LOS and number of patients readmitted to the ICU were similar in both groups, it is unlikely that the postintervention group represented a less-sick cohort.

Important Findings and Implications

This QI project highlighted novel ways to implement and emphasize the importance of compliance to CPG. A provider pledge may be helpful in reinforcing to all providers the idea that the institution is committed to guideline implementation. Comparing individual provider data and having a real-time dashboard with group performance can help reinforce goals and progress toward them at the group, site, and individual patient population levels.

Development and Validation of a Calculator for Estimating the Probability of Urinary Tract Infection in Young Febrile Children. Shaikh N, et al. JAMA Pediatrics. 2018;172(6):550-556.³

Background

The prevalence of urinary tract infections (UTIs) in children under 2 years of age that present to the emergency department (ED) with fever is about 7%.⁴ After clinical examination, providers obtaining a urinalysis must then determine if empirical antibiotics are warranted for a suspected UTI. This study describes the development of a novel calculator, UTICalc that estimates the pretest probability of a UTI based on clinical findings and the posttest probability of a UTI based on laboratory results.

Study Overview and Results

This study features a single center, nested, case-control design that looked retrospectively at 542 children aged 2-24 months who presented to the ED from January 2007 to April 2013 with fever and had a catheterized urinalysis obtained. Patients were then matched with randomly selected children without a UTI to create a training database. Five models using different variables were developed, including one with only clinical characteristics and four that combined clinical characteristics with differing laboratory values. The area under the curve of the “clinical model” was 0.80, while those of the remaining four models ranged from 0.97 to 0.98. The clinical model showed a sensitivity of 95% and specificity of 35% in the training database, while the four other models showed sensitivities ranging from 93% to 96% and specificities ranging from 91% to 93%. The models were then validated using a cohort of children aged 2-24 months who presented to the ED with fever from July 2015 to December 2016; the UTI prevalence in this cohort was 7.8%. Finally, using a hypothetical cohort of 1,000 children being evaluated for a UTI, the authors showed that UTICalc reduced the numbers of urine samples obtained by 8.1% and missed UTIs from 3 to 0 compared with following AAP guidelines.⁵

Limitations

The training database was created retrospectively at a single institution and is subject to local practice patterns. The proposed calculator creates an algorithm that is meant to be used in a setting where the pretest probability for a UTI is reasonably high based on criteria from the AAP UTI guidelines.

Important Findings and Implications

UTICalc could be a great tool for providers to guide testing for UTIs in children aged 2-24 months presenting with a fever. Given further study at multiple sites and settings, including outpatient clinics, UTICalc could have significant implications for reducing unnecessary testing and treatment in febrile children.

Lost Earnings and Nonmedical Expenses of Pediatric Hospitalizations. Chang LV, et al. Pediatrics. 2018;142(3):e20180195.⁶

Background

Although medical expenses related to hospitalization can be significant for many families, nonmedical costs, such as transportation, parking, meals, and lost earnings from missed days at work, are also important to consider. These hardships can lead to challenges in postdischarge follow-up and adherence to discharge instructions, both of which lead to hospital readmissions. This article presents a cross-sectional analysis at a large, free-standing children’s hospital that participated in the Hospital-to-Home Outcomes Study (H2O). The authors sought to determine whether families with more financial or social hardships are affected disproportionately by nonmedical costs related to hospitalizations.

Study Overview and Results

A total of 1,372 children were included and children with lengths of stay >13 days were excluded. Face-to-face parental surveys were conducted and included questions on parental education, employment status, sick leave flexibility, and measures of financial and social hardship. The study authors calculated a total cost burden (TCB) based on nonmedical costs estimated at the time of the survey, including lost wages and expenses during the hospitalization. A daily cost burden (DCB) based on length of hospital stay and daily cost burden as a percentage of daily income (DCBi) were also calculated. The median TCB was $112.80, and the median DCB was $51.40. The median DCBi showed that the median household had 45% of their daily income depleted by nonmedical expenses related to their hospitalization. Those who reported more financial or social hardships had a higher median DCBi; if ≥3 financial hardships were reported, 86% of the daily household income was depleted.

Limitations

The study was conducted at a single institution with a number of existing support systems in place to help unburden families of hospitalized children. Non-English-speaking families were excluded. A face-to-face survey may have influenced parental responses regarding social and financial hardships.

Important Findings and Implications

Nonmedical costs of hospitalized children can be quantified and disproportionately affect those experiencing financial and social hardships. Hospitalists should be aware of these findings and find ways within their hospital systems to provide support for families both during and after hospitalizations.

A Prescription for Note Bloat: An Effective Progress Note Template. Kahn D, et al. Journal of Hospital Medicine. 2018;13(6):378-382.⁷

Background

Although electronic health records (EHRs) have improved the speed and legibility of documentation, the harm of “note bloat,” defined as multiple pages of nonessential information which leaves key aspects buried or lost, is prevalent. In this prospective, quality improvement study across four internal medicine residency programs, the authors investigated a bundled intervention consisting of didactic teaching and an electronic progress note template on note quality, length, and timeliness.

Study overview and results

Notes pre- and postintervention were graded using a tool that considered the general impression of the note, its score on the validated Physician Documentation Quality Instrument (PDQI-9),⁸ and a questionnaire based on the Accreditation Council for Graduate Medical Education competency note checklist.⁹ Analyzing 200 preintervention and 199 postintervention notes, significant improvement was seen in general impression scores, all PDQI-9 domains, and 6 of 13 note competency questionnaire items. The mean number of lines in the note decreased by 25%, and the mean completion time when the note was signed was 1 hour and 15 minutes earlier. The greatest impact on shortening notes involved a reduction in the auto-population of laboratory and imaging studies.

Limitations

The study was unblinded. The authors attempted to minimize bias with an objective questionnaire and employed multiple graders per note; however, poor interrater reliability was obtained. Postintervention, 70% of all residents used the template. At one of the four institutions, evidence of note quality improvement despite low template use was found. At another institution, no improvement in note quality was reported despite relatively high template uptake. Local culture and institutional buy-in may be factors affecting these results. In addition, pre- and postintervention notes were examined in the same academic year; thus, the effects seen may be due, in part, to resident maturation. Generalizability to nonacademic institutions and the durability of the intervention are additional concerns.

Important Findings and Implications

Resident education on documentation and an EHR progress note template incorporating best practices can effectively combat “note bloat” and lead to higher quality and shorter notes that are completed earlier in the day. This solution has significant implications for improving transitions of care, handoffs, and patient safety.

Time to Pathogen Detection for Non-Ill Versus Ill-Appearing Infants ≤60 Days Old with Bacteremia and Meningitis. Aronson PL, et al. Hospital Pediatrics 2018;8 (7):379-384.¹⁰

Background

The routine evaluation of febrile infants aged ≤60 days old often involves blood and cerebrospinal (CSF) fluid evaluations, and many infants are hospitalized while waiting for culture results. A previous study of febrile infants showed that 91% of the pathogenic organisms could be identified on blood culture within 24 hours and that 96% could be identified within 36 hours; 81% of the bacterial pathogens present were detected on CSF culture within 36 hours.¹¹

Study Overview and Results

In this large, multicenter study of infants presenting to the Emergency Departments (EDs) of 10 children’s hospitals over a five-year study period, the authors investigated the time to pathogen detection in blood and CSF for infants aged ≤60 days with bacteremia and/or bacterial meningitis; whether the time to detection differed for non-ill and ill infants was also examined. Ill- versus non-ill-appearance was determined by a medical record review of the physical exam looking for one of 13 key words (eg, “ill-appearing,” “toxic,” “lethargic,” etc.). A total of 381 infants were included. Overall, 88% of the pathogens present were detected in blood culture within 24 hours and 95% were detected within 36 hours. In CSF, 89% of the pathogens present were detected within 24 hours, and 95% were detected within 36 hours. In infants with bacteremia who were non-ill-appearing, 85% of the blood pathogens were detected within 24 hours.

Limitations

The median time to detection for blood culture pathogens for ill-appearing versus non-ill-appearing infants was shorter by just one hour, but 15% of the non-ill infants had a positive blood culture after 24 hours. However, the prevalence of bacteremia and meningitis in non-ill-appearing infants is likely low; the authors did not report the total number of febrile infants evaluated by EDs in the study.

Important Findings and Implications

Most positive blood and/or CSF cultures for infants aged ≤60 days will yield results by 24 hours; 95% of the pathogens present could be detected within 36 hours. Sending a non-ill-appearing febrile infant home at 24 hours may miss 15% of the instances of bacteremia, but the overall low prevalence of invasive bacterial infection in infants should be considered.

The High-Value Care Rounding Tool: Development and Validity Evidence. McDaniel CE, et al. Academic Medicine. 2018;93(2):199-206.¹²

Background

Providing high-value care (HVC) to patients is a struggle for physicians and healthcare systems. Although physicians teaching trainees HVC practices could be an effective way to increase cost-conscious care, the best practices for teaching HVC remain unknown. To fill this gap, the authors developed a tool to measure the frequency and content of observable HVC teaching and evaluated the validity of the tool within a pediatric inpatient setting.

Study Overview and Results

The HVC rounding tool was developed through several phases from conception to validation. The research group used a modified Delphi method to construct the tool using a consensus building process based on opinions from content experts in the field of HVC, from a variety of specialties, experience levels, and geographic areas of the United States. Each item of the HVC instrument was rated by these experts, and, from their evaluations and surveys, an 11-item HVC tool was constructed. A pilot of the tool was performed to establish internal validity and interrater reliability based on observations of 148 patient encounters. From this process, a final 10-item HVC rounding tool emerged, including domains in quality, cost, and patient values. A few items included giving positive feedback for not doing an unnecessary test, discussing whether a patient needs to stay inpatient or meets discharge criteria, and customizing a care plan to align with family values and goals. The final iteration of the tool had no rater disagreements within the quality and patient values domain and only one disagreement within the cost domain.

Limitations

This tool was validated at a single pediatric institution, and, thus, the generalizability of the tool has not been established. The authors note that the Delphi panelists used for the construction of the tool were from a medical subspecialty background and not surgical backgrounds, which limits its applicability from a surgical perspective. The tool does not allow for differentiation between lengthy discussions or brief comments presented during rounds.

Important Findings and Implications

The HVC rounding tool is both innovative and timely. Pediatric hospitalists are leaders in family-centered care, and this tool allows assessment of whether important concepts of high-value care are discussed at the bedside. A multisite educational study using this tool would be welcome.

Disclosures

The authors have no conflicts of interest to declare.

The field of pediatric hospital medicine has seen tremendous growth in scholarship in the past decade. To obtain a wide view of advancements in the field from the current literature, the authors selected 18 English-language journals (Table 1) across four domains believed to be relevant to the practice of pediatric hospital medicine, including hospital medicine, pediatrics, emergency care, and medical education. The median Hirsch index (h-index) of the selected journals was 131. A goal of 10, a number that could maximally benefit consumers of the finished product, was set as the final number of articles to be selected.

Guiding principles for the initial selection included novelty of hypotheses, study design, significance of results, and likelihood to change pediatric hospital medicine practice from both the community and academic hospital perspectives. Journals were assigned randomly to each author for review and assignments were switched after six months to limit potential bias in coverage. A three-stage review process was employed. The authors initially independently reviewed titles and abstracts from 13,296 articles published between January 2018 and December 2018 and rated them according to their likelihood to be included in the final set of 10 articles and their broad applicability to pediatric hospital medicine. This resulted in 99 studies that were selected for further review. Next, the authors were assigned a subset of the 99 articles for further review; each author rated the articles independently based on their likelihood of inclusion in the final 10-article set. At this stage, 75 articles were excluded. Finally, all remaining 24 articles were reviewed independently and in depth by both authors.

Ten articles were selected by consensus formation, and the authors presented their findings at the 2019 Society for Hospital Medicine annual meeting. From these 10 articles, six were determined to be most impactful to current practice; these articles are presented below. After discussing the study background, an overview, key results, limitations of the study, important findings (Table 2), and implications for practice and future research are presented.

Interventions to Reduce Over-Utilized Tests and Treatments in Bronchiolitis. Tyler A, et al. Pediatrics. 2018;141(6):e20170485. ¹

Background

The American Academy of Pediatrics (AAP) published clinical practice guidelines (CPG) for bronchiolitis in 2014.² However, unnecessary tests and interventions continue to be ordered and used on children with bronchiolitis that are not recommended by the guidelines. In this quality improvement project, the authors sought to increase compliance with the AAP CPG for bronchiolitis by reducing chest x-rays (CXR) to <20%, respiratory viral testing (RVT) to <15%, and use of bronchodilators to <20%.

Study Overview and Results

This project took place at a free-standing children’s hospital and included urgent care locations. Authors obtained pre-intervention data through two bronchiolitis seasons in 2013 and 2014 for patients aged 1-23 months with a primary or secondary diagnosis of bronchiolitis and who did not require admission to the Intensive Care Unit (ICU). The intervention period was from December 2015 to April 2016. All sites simultaneously implemented their interventions, which included education of care team members and families, updated order sets, and electronic health record (EHR)-generated e-mails that provided data looking at peer ranking statistics for each intervention, CXR, RVT, and bronchodilator usage. A data dashboard was created to display real-time utilization of the studied interventions. Providers were also asked to sign a pledge that they would reduce unnecessary testing and treatment. As balancing measures, the numbers of patients presenting to the Emergency Room (ER) or readmitted within seven days of an ED visit or admission for bronchiolitis were tracked; patients who required ICU levels of care during their first admission or on readmission were also tracked. Statistically significant decreases in CXR ordering from 39.5% to 27.2%, RVT ordering from 31.9% to 26.3%, and any bronchodilator usage from 34.2% to 21.5% were noted. No difference pre- and postintervention in patients readmitted to the ICU was found, and length of stay (LOS) between groups was not statistically significant.

Limitations

As all interventions were initiated simultaneously, identifying which individual or subset of interventions was responsible for changing provider behavior was impossible. More patients postintervention were admitted under observation status and under a milder All Patient Refined Diagnosis Related Groups (APR DRGs) severity index, which may indicate a less-sick cohort of patients in this group. Since the LOS and number of patients readmitted to the ICU were similar in both groups, it is unlikely that the postintervention group represented a less-sick cohort.

Important Findings and Implications

This QI project highlighted novel ways to implement and emphasize the importance of compliance to CPG. A provider pledge may be helpful in reinforcing to all providers the idea that the institution is committed to guideline implementation. Comparing individual provider data and having a real-time dashboard with group performance can help reinforce goals and progress toward them at the group, site, and individual patient population levels.

Development and Validation of a Calculator for Estimating the Probability of Urinary Tract Infection in Young Febrile Children. Shaikh N, et al. JAMA Pediatrics. 2018;172(6):550-556.³

Background

The prevalence of urinary tract infections (UTIs) in children under 2 years of age that present to the emergency department (ED) with fever is about 7%.⁴ After clinical examination, providers obtaining a urinalysis must then determine if empirical antibiotics are warranted for a suspected UTI. This study describes the development of a novel calculator, UTICalc that estimates the pretest probability of a UTI based on clinical findings and the posttest probability of a UTI based on laboratory results.

Study Overview and Results

This study features a single center, nested, case-control design that looked retrospectively at 542 children aged 2-24 months who presented to the ED from January 2007 to April 2013 with fever and had a catheterized urinalysis obtained. Patients were then matched with randomly selected children without a UTI to create a training database. Five models using different variables were developed, including one with only clinical characteristics and four that combined clinical characteristics with differing laboratory values. The area under the curve of the “clinical model” was 0.80, while those of the remaining four models ranged from 0.97 to 0.98. The clinical model showed a sensitivity of 95% and specificity of 35% in the training database, while the four other models showed sensitivities ranging from 93% to 96% and specificities ranging from 91% to 93%. The models were then validated using a cohort of children aged 2-24 months who presented to the ED with fever from July 2015 to December 2016; the UTI prevalence in this cohort was 7.8%. Finally, using a hypothetical cohort of 1,000 children being evaluated for a UTI, the authors showed that UTICalc reduced the numbers of urine samples obtained by 8.1% and missed UTIs from 3 to 0 compared with following AAP guidelines.⁵

Limitations

The training database was created retrospectively at a single institution and is subject to local practice patterns. The proposed calculator creates an algorithm that is meant to be used in a setting where the pretest probability for a UTI is reasonably high based on criteria from the AAP UTI guidelines.

Important Findings and Implications

UTICalc could be a great tool for providers to guide testing for UTIs in children aged 2-24 months presenting with a fever. Given further study at multiple sites and settings, including outpatient clinics, UTICalc could have significant implications for reducing unnecessary testing and treatment in febrile children.

Lost Earnings and Nonmedical Expenses of Pediatric Hospitalizations. Chang LV, et al. Pediatrics. 2018;142(3):e20180195.⁶

Background

Although medical expenses related to hospitalization can be significant for many families, nonmedical costs, such as transportation, parking, meals, and lost earnings from missed days at work, are also important to consider. These hardships can lead to challenges in postdischarge follow-up and adherence to discharge instructions, both of which lead to hospital readmissions. This article presents a cross-sectional analysis at a large, free-standing children’s hospital that participated in the Hospital-to-Home Outcomes Study (H2O). The authors sought to determine whether families with more financial or social hardships are affected disproportionately by nonmedical costs related to hospitalizations.

Study Overview and Results

A total of 1,372 children were included and children with lengths of stay >13 days were excluded. Face-to-face parental surveys were conducted and included questions on parental education, employment status, sick leave flexibility, and measures of financial and social hardship. The study authors calculated a total cost burden (TCB) based on nonmedical costs estimated at the time of the survey, including lost wages and expenses during the hospitalization. A daily cost burden (DCB) based on length of hospital stay and daily cost burden as a percentage of daily income (DCBi) were also calculated. The median TCB was $112.80, and the median DCB was $51.40. The median DCBi showed that the median household had 45% of their daily income depleted by nonmedical expenses related to their hospitalization. Those who reported more financial or social hardships had a higher median DCBi; if ≥3 financial hardships were reported, 86% of the daily household income was depleted.

Limitations

The study was conducted at a single institution with a number of existing support systems in place to help unburden families of hospitalized children. Non-English-speaking families were excluded. A face-to-face survey may have influenced parental responses regarding social and financial hardships.

Important Findings and Implications

Nonmedical costs of hospitalized children can be quantified and disproportionately affect those experiencing financial and social hardships. Hospitalists should be aware of these findings and find ways within their hospital systems to provide support for families both during and after hospitalizations.

A Prescription for Note Bloat: An Effective Progress Note Template. Kahn D, et al. Journal of Hospital Medicine. 2018;13(6):378-382.⁷

Background

Although electronic health records (EHRs) have improved the speed and legibility of documentation, the harm of “note bloat,” defined as multiple pages of nonessential information which leaves key aspects buried or lost, is prevalent. In this prospective, quality improvement study across four internal medicine residency programs, the authors investigated a bundled intervention consisting of didactic teaching and an electronic progress note template on note quality, length, and timeliness.

Study overview and results

Notes pre- and postintervention were graded using a tool that considered the general impression of the note, its score on the validated Physician Documentation Quality Instrument (PDQI-9),⁸ and a questionnaire based on the Accreditation Council for Graduate Medical Education competency note checklist.⁹ Analyzing 200 preintervention and 199 postintervention notes, significant improvement was seen in general impression scores, all PDQI-9 domains, and 6 of 13 note competency questionnaire items. The mean number of lines in the note decreased by 25%, and the mean completion time when the note was signed was 1 hour and 15 minutes earlier. The greatest impact on shortening notes involved a reduction in the auto-population of laboratory and imaging studies.

Limitations

The study was unblinded. The authors attempted to minimize bias with an objective questionnaire and employed multiple graders per note; however, poor interrater reliability was obtained. Postintervention, 70% of all residents used the template. At one of the four institutions, evidence of note quality improvement despite low template use was found. At another institution, no improvement in note quality was reported despite relatively high template uptake. Local culture and institutional buy-in may be factors affecting these results. In addition, pre- and postintervention notes were examined in the same academic year; thus, the effects seen may be due, in part, to resident maturation. Generalizability to nonacademic institutions and the durability of the intervention are additional concerns.

Important Findings and Implications

Resident education on documentation and an EHR progress note template incorporating best practices can effectively combat “note bloat” and lead to higher quality and shorter notes that are completed earlier in the day. This solution has significant implications for improving transitions of care, handoffs, and patient safety.

Time to Pathogen Detection for Non-Ill Versus Ill-Appearing Infants ≤60 Days Old with Bacteremia and Meningitis. Aronson PL, et al. Hospital Pediatrics 2018;8 (7):379-384.¹⁰

Background

The routine evaluation of febrile infants aged ≤60 days old often involves blood and cerebrospinal (CSF) fluid evaluations, and many infants are hospitalized while waiting for culture results. A previous study of febrile infants showed that 91% of the pathogenic organisms could be identified on blood culture within 24 hours and that 96% could be identified within 36 hours; 81% of the bacterial pathogens present were detected on CSF culture within 36 hours.¹¹

Study Overview and Results

In this large, multicenter study of infants presenting to the Emergency Departments (EDs) of 10 children’s hospitals over a five-year study period, the authors investigated the time to pathogen detection in blood and CSF for infants aged ≤60 days with bacteremia and/or bacterial meningitis; whether the time to detection differed for non-ill and ill infants was also examined. Ill- versus non-ill-appearance was determined by a medical record review of the physical exam looking for one of 13 key words (eg, “ill-appearing,” “toxic,” “lethargic,” etc.). A total of 381 infants were included. Overall, 88% of the pathogens present were detected in blood culture within 24 hours and 95% were detected within 36 hours. In CSF, 89% of the pathogens present were detected within 24 hours, and 95% were detected within 36 hours. In infants with bacteremia who were non-ill-appearing, 85% of the blood pathogens were detected within 24 hours.

Limitations

The median time to detection for blood culture pathogens for ill-appearing versus non-ill-appearing infants was shorter by just one hour, but 15% of the non-ill infants had a positive blood culture after 24 hours. However, the prevalence of bacteremia and meningitis in non-ill-appearing infants is likely low; the authors did not report the total number of febrile infants evaluated by EDs in the study.

Important Findings and Implications

Most positive blood and/or CSF cultures for infants aged ≤60 days will yield results by 24 hours; 95% of the pathogens present could be detected within 36 hours. Sending a non-ill-appearing febrile infant home at 24 hours may miss 15% of the instances of bacteremia, but the overall low prevalence of invasive bacterial infection in infants should be considered.

The High-Value Care Rounding Tool: Development and Validity Evidence. McDaniel CE, et al. Academic Medicine. 2018;93(2):199-206.¹²

Background

Providing high-value care (HVC) to patients is a struggle for physicians and healthcare systems. Although physicians teaching trainees HVC practices could be an effective way to increase cost-conscious care, the best practices for teaching HVC remain unknown. To fill this gap, the authors developed a tool to measure the frequency and content of observable HVC teaching and evaluated the validity of the tool within a pediatric inpatient setting.

Study Overview and Results

The HVC rounding tool was developed through several phases from conception to validation. The research group used a modified Delphi method to construct the tool using a consensus building process based on opinions from content experts in the field of HVC, from a variety of specialties, experience levels, and geographic areas of the United States. Each item of the HVC instrument was rated by these experts, and, from their evaluations and surveys, an 11-item HVC tool was constructed. A pilot of the tool was performed to establish internal validity and interrater reliability based on observations of 148 patient encounters. From this process, a final 10-item HVC rounding tool emerged, including domains in quality, cost, and patient values. A few items included giving positive feedback for not doing an unnecessary test, discussing whether a patient needs to stay inpatient or meets discharge criteria, and customizing a care plan to align with family values and goals. The final iteration of the tool had no rater disagreements within the quality and patient values domain and only one disagreement within the cost domain.

Limitations

This tool was validated at a single pediatric institution, and, thus, the generalizability of the tool has not been established. The authors note that the Delphi panelists used for the construction of the tool were from a medical subspecialty background and not surgical backgrounds, which limits its applicability from a surgical perspective. The tool does not allow for differentiation between lengthy discussions or brief comments presented during rounds.

Important Findings and Implications

The HVC rounding tool is both innovative and timely. Pediatric hospitalists are leaders in family-centered care, and this tool allows assessment of whether important concepts of high-value care are discussed at the bedside. A multisite educational study using this tool would be welcome.

Disclosures

The authors have no conflicts of interest to declare.

A 22-year-old man presented to a Canadian community hospital emergency department complaining of 2-3 weeks of abdominal pain and bloating associated with early satiety. He also noted weight loss of 20 pounds over the preceding months, leg and abdominal swelling with increased girth, and 1-2 loose, nonbloody stools per day.

Early satiety and bloating are nonspecific symptoms that can be due to gastroesophageal reflux disease, peptic ulcer disease, gastrointestinal obstruction, or gastroparesis. Weight loss in a young person, particularly if >5% of body weight, is concerning for a serious underlying medical issue. It could reflect reduced intake due to anorexia, odynophagia, or dysphagia or increased energy expenditure due to an inflammatory state such as infection or rheumatic disease. The etiology of the swelling needs to be elucidated. It may be due to increased hydrostatic forces as in heart failure, venous or lymphatic obstruction, or from lowered oncotic pressure resulting from hepatic disease, nephrotic syndrome, severe malnutrition (nonbloody loose stools), or a protein losing enteropathy.

The patient was transferred to a tertiary care center for closer access to specialty consultation. He described generalized abdominal pain increasing in intensity over three weeks; bilateral lower extremity, scrotal, abdominal wall, and sacral edema; and mild dyspnea on exertion. The early satiety was not associated with dysphagia, odynophagia, nausea, or vomiting. He denied fevers, chills, night sweats, nausea, vomiting, jaundice, easy bruising, orthopnea, paroxysmal nocturnal dyspnea (PND), or chest pain. His past medical history included asthma treated with fluticasone/salmeterol and albuterol. He was a Canadian of East Asian descent working as a plumber. He previously smoked three to four cigarettes per day for six years. He stopped smoking one month before presentation. He had one alcoholic beverage per week and smoked marijuana weekly. He denied any family history of similar symptoms or malignancy.The differential diagnosis for weight loss and anasarca is broad and includes malignancies, infectious diseases, rheumatic or inflammatory disorders, malabsorption, and advanced cardiac, renal, or liver disease. His history does not classically point in one direction. The mild dyspnea on exertion may be due to cardiac disease, but it is unlikely in the absence of orthopnea and PND. The dyspnea could be due to increased abdominal pressure if ascites are present, his underlying asthma, or another etiology such as anemia. Fevers, chills, and/or night sweats can be expected in infections and some malignancies, but their absence does not exclude infections and malignancies from the differential diagnoses. Particular attention should be paid to lymphadenopathy on the physical examination. The presence of an umbilical nodule (Sister Mary Joseph sign) could indicate a malignancy (gastrointestinal or lymphoma).

The differential diagnosis for weight loss and anasarca is broad and includes malignancies, infectious diseases, rheumatic or inflammatory disorders, malabsorption, and advanced cardiac, renal, or liver disease. His history does not classically point in one direction. The mild dyspnea on exertion may be due to cardiac disease, but it is unlikely in the absence of orthopnea and PND. The dyspnea could be due to increased abdominal pressure if ascites are present, his underlying asthma, or another etiology such as anemia. Fevers, chills, and/or night sweats can be expected in infections and some malignancies, but their absence does not exclude infections and malignancies from the differential diagnoses. Particular attention should be paid to lymphadenopathy on the physical examination. The presence of an umbilical nodule (Sister Mary Joseph sign) could indicate a malignancy (gastrointestinal or lymphoma).

On physical examination, his temperature was 38.1°C, heart rate was 138 beats per minute, blood pressure was 123/86 mm Hg, respiratory rate was 20 breaths per minute, and oxygen saturation was 97% on room air. He appeared uncomfortable and diaphoretic. No scleral icterus or jaundice was appreciated. There were no palpable cervical, axillary, or inguinal lymph nodes. Cardiac examination revealed tachycardia and no murmurs, rubs, gallops, or jugular venous distention. Abdominal examination revealed abdominal distention, diffuse tenderness to deep palpation, bulging flanks, and a positive fluid wave. Liver and spleen could not be palpated or percussed secondary to abdominal distention. He had pitting bilateral lower extremity edema that extended to and included the scrotum. Neurologic and pulmonary examinations were unremarkable.

His examination reveals low-grade fever, tachycardia, and diaphoresis. Whether this represents progression of his primary disease or he has acutely developed a superimposed infection is uncertain at this point. He has notable anasarca but no jugular venous distention, crackles, or S3 gallop. The lack of evidence of pulmonary edema or increased central venous pressure on physical examination increases the likelihood of cirrhosis, hypoalbuminemia, or obstruction (lymphatic or venous) and decreases the likelihood of heart failure as the etiology of his peripheral edema and likely ascites. Despite the prominence of gastrointestinal symptoms, he has neither jaundice nor stigmata of chronic liver disease. Periorbital edema, which may be present in nephrotic syndrome, is also absent. Although he has no palpable peripheral lymphadenopathy, malignancy remains a concern.

Testing should include urinalysis for proteinuria and coagulation studies to assess synthetic function of the liver. Abdominal ultrasound is indicated to confirm ascites. If present, a diagnostic paracentesis should be performed to rule out spontaneous bacterial peritonitis and determine whether the ascites is from portal hypertension, hypoalbuminemia, or peritoneal disease. If the transaminases are elevated or if the ascitic fluid is concerning for malignancy, he will need a computed tomography (CT) of the abdomen and pelvis. A protein losing enteropathy due to malignancies (gastric cancer or lymphoma), rheumatic disease (systemic lupus erythematosus [SLE]), or infiltrative disease (amyloid) is also a possibility. If the other studies are unrevealing, stool should be sent for alpha-1 antitrypsin.

Laboratory studies revealed hemoglobin 7.8 g/dL, platelets 53 k/mm³, white blood cell count (WBC) 10.6 k/mm³, alkaline phosphatase (ALP) 217 U/L, albumin 2.7 g/dL, reticulocyte count 3 k/mm³ (reference range, 30-110 k/mm³), and ferritin 1,310 ng/mL (reference range, 20-400 ng/L). Serum aminotransferase levels, bilirubin, coagulation panel, electrolytes, and creatinine were normal. Urinalysis was negative for blood, leukocytes, and protein. Diagnostic paracentesis demonstrated a serum-ascites-albumin gradient (SAAG) of two and macrophage predominance (WBC 250 U/L). Ascitic fluid cytology and culture were negative. Blood cultures, human immunodeficiency virus (HIV)-1 and 2, cytomegalovirus (CMV), and Epstein–Barr virus (EBV) serologies were negative. Viral serologies for hepatitis A, B, and C were negative. Antinuclear antibody (ANA), anti-ds DNA, antineutrophilic cytoplasmic antibody (ANCA), serum angiotensin-converting enzyme (ACE) level, and quantitative immunoglobulin levels were all within the normal range. Chest, abdomen, and pelvis CT with contrast revealed large-volume abdominal and pelvic ascites, diffuse subcutaneous edema (Figure 1), modest hepatosplenomegaly, small bilateral pleural effusions, and mediastinal, axillary, mesenteric, periportal, peripancreatic, and retroperitoneal lymphadenopathy (Figure 2).

Some of the more common viral etiologies of generalized lymphadenopathy and cytopenias are unlikely because serologies for HIV, hepatitis B and C, EBV, and CMV are negative. Tuberculosis fits with the insidious nature of his presentation and remains on the differential although a low SAAG would be expected. From a rheumatologic standpoint, the lack of characteristic findings on history and physical examination and the negative ANA and anti-ds DNA results make SLE unlikely. Although elevated in the majority of untreated sarcoid patients, a normal ACE level is not sufficient to rule out this diagnosis. IgG, IgA, and IgM levels would be low if there was significant gastrointestinal protein loss and elevated in MCD. The markedly increased ferritin level, an acute-phase reactant often elevated in the setting of inflammation or malignancy, raises suspicion for adult Still’s disease (despite the lack of characteristic arthralgias and/or rash) and hemophagocytic lymphohistiocytosis (HLH).

A SAAG greater than or equal to 1.1 indicates the presence of portal hypertension. Portal hypertension most often results from cirrhosis for which this patient has no apparent clinical findings. Etiologies of noncirrhotic portal hypertension are classified as prehepatic, intrahepatic, and posthepatic. There is no clinical or radiologic evidence of portal or splenic vein thrombosis (prehepatic) or heart failure (posthepatic). Possible intrahepatic etiologies include malignancy and sarcoid. Although uncommon, patients with malignancy-related ascites may have a high SAAG without coexisting cirrhosis. This occurs if there is portal hypertension due to extensive metastases in the liver or involvement of the portal venous system. The cytology of the ascitic fluid is negative. However, cytology is <80% sensitive in the absence of peritoneal carcinomatosis.

Hematology was consulted for evaluation of the lymphadenopathy, anemia, and thrombocytopenia and recommended bone marrow and excisional lymph node biopsies. Bone marrow biopsy showed trilineage hypercellularity (Figure 3A) with reduced erythropoiesis and reticulin fibrosis (Figure 3B). An axillary lymph node biopsy with flow cytometry was nondiagnostic for a lymphoproliferative disorder or malignancy.

Five days after discharge, he was readmitted with worsening anasarca, massive ascites, and acute kidney injury. Admission laboratory studies revealed creatinine 1.66 mg/dL, hemoglobin 11.5 g/dL, and platelets 94 k/mm³. In addition, his ferritin level was 1,907 ng/L (reference range, 20-400 ng/L), erythrocyte sedimentation rate (ESR) was 50 mm/h (reference range, 0-20 mm/h), and C-reactive protein concentration (CRP) was 12.1 mg/dL (reference range, 0-0.5 mg/dL).

Steroids are used to treat a wide variety of illnesses, some of which are still under consideration in this patient including lymphoma, MCD, adult Still’s disease, and HLH. His symptoms recurred quickly after discontinuation of steroids in the setting of elevated ferritin, ESR, and CRP levels reflecting marked ongoing inflammation. Serologic testing for soluble IL-2 receptor, often elevated in MCD and HLH, should be performed. Excisional biopsy of an accessible node should be performed urgently.

His acute kidney injury resolved; however, he continued to have intermittent fevers, anemia, thromobocytopenia, lymphadenopathy, and hepatosplenomegaly. A hematology case-conference recommended testing for HLH, including soluble IL-2 receptor (CD25), soluble CD163, and natural killer cell degranulation assay, all of which were negative. A right inguinal lymph node biopsy revealed reactive lymphoid tissue and stained negative for HHV-8. Based on the lack of an alternative diagnosis (particularly lymphoma), the presence of multiple areas of lymphadenopathy, anemia, fevers, organomegaly, weight loss, reactive lymphoid tissue on lymph node biopsy, and elevated CRP and ESR, a working diagnosis of MCD was made. The negative HHV-8 testing was consistent with idiopathic MCD (iMCD); however, features inconsistent with iMCD included lack of polyclonal hypergammaglobulinemia and the presence of significant anasarca and thrombocytopenia. Therefore, an internet search was performed using the patient’s salient symptoms and findings. The search revealed a few recently published case reports of a rare variant of iMCD, TAFRO syndrome. TAFRO syndrome, characterized by thrombocytopenia, anasarca, fever, reticulin fibrosis and/or renal insufficiency, and organomegaly, fully explained the patient’s presentation. He was started on prednisone, rituximab (anti-CD20 antibody), and furosemide. After one month of treatment, he showed complete resolution of cytopenias, lymphadenopathy, organomegaly, anasarca, and ascites. Therapy continued for approximately three months, and he has remained symptom-free.

Castleman’s disease (CD) is a rare lymphoproliferative disorder divided into unicentric (solitary enlarged lymph node) and multicentric (multifocal enlarged lymph nodes).¹ MCD typically presents with systemic inflammation, reactive proliferation of benign lymphocytes, multifocal lymphadenopathy, elevated inflammatory markers, anemia, hypoalbuminemia, and polyclonal gammaglobulinemia.¹ It is hypothesized that HHV-8 drives the systemic inflammation of MCD via high levels of interleukin-6 (IL-6) activity.¹ iMCD is an HHV-8-negative variant of MCD.¹

TAFRO syndrome was first described in 2010 in three Japanese patients demonstrating high fever, anasarca, hepatosplenomegaly, lymphadenopathy, severe thrombocytopenia, and reticulin fibrosis.² In 2015, the All Japan TAFRO Syndrome Research Group recognized TAFRO syndrome as a variant of iMCD and created diagnostic criteria and a severity classification system.³ Major criteria consist of anasarca, including pleural effusion and/or ascites identified on CT scan and general edema, thrombocytopenia (platelet count <100 k/mm³), and systemic inflammation (fever >37.5°C and/or serum CRP greater than or equal to 2 mg/dL).³ Two of four minor criteria must be met, which include (1) lymph node histology consistent with CD, (2) reticulin myelofibrosis and/or increased number of megakaryocytes in bone marrow, (3) mild organomegaly, including hepatomegaly, splenomegaly, and lymphadenopathy <1.5 cm in diameter identified on CT scan, and (4) progressive renal insufficiency (serum creatinine >1.2 mg/dL in males or >1.0 mg/dL in females).³ In addition, several patients with TAFRO syndrome demonstrate elevated ALP, low-normal LDH, elevated vascular endothelial growth factor, elevated IL-6, microcytic anemia, and slight polyclonal hypergammopathy.³ Malignancies such as lymphoma and myeloma, autoimmune diseases such as SLE and ANCA-associated vasculitis, infectious diseases such as those caused by mycobacteria, and POEMS (polyneuropathy, organomegaly, endocrine diseases, M-protein, and skin lesions) syndrome must be excluded to diagnose TAFRO syndrome.^3,4

The pathophysiology of TAFRO syndrome is unknown, and it is unclear whether the syndrome is truly a variant of iMCD or a distinct entity.³ IL-6 is typically only mildly elevated in TAFRO syndrome, without the consequent thrombocytosis and polyclonal hypergammaglobulinemia seen in MCD, which is associated with higher levels of IL-6.¹ Multiple non-HHV-8 mechanisms for TAFRO syndrome have been proposed, including (1) systemic inflammation, autoimmune/autoinflammatory mechanisms, (2) neoplastic, ectopic cytokine secretion by malignant or benign tumor cells, and/or (3) infectious, such as non-HHV-8 virus.⁵

Immunosuppression is the mainstay of treatment for TAFRO syndrome based on recommendations from the 2015 TAFRO Research Group.³ Glucocorticoids are considered first-line therapy.³ Cyclosporin A is recommended for individuals refractory to glucocorticoids.³ In patients with a contraindication to cyclosporin A, anti-IL-6 receptor antibodies such as tocilizumab (approved for treatment of iMCD in Japan) and siltuximab (approved for treatment of iMCD in North America and Europe) or the anti-CD20 antibody rituximab should be prescribed.³ There is evidence for the thrombopoietin receptor agonists romiplostim and eltrombopag to treat persistent thrombocytopenia.³ Additional treatments for refractory TAFRO syndrome include IVIG and plasma exchange, chemotherapy (cyclophosphamide, doxorubicin, vincristine, prednisolone), and thalidomide.^3,6

Little is known about the epidemiologic characterization of TAFRO syndrome as less than 40 cases of TAFRO syndrome have been reported in the United States, Asia, and Europe.

• Key clinical and pathologic features of TAFRO syndrome include thrombocytopenia, anasarca, fever, reticulin fibrosis and/or renal insufficiency, and organomegaly.
• TAFRO syndrome may be under-recognized due to very recent characterization and no previous ICD code for CD.
• TAFRO syndrome experts recommend immunosuppression for treatment of TAFRO syndrome, including glucocorticoids as first-line treatment.
• Internet searches can be helpful in the diagnosis of challenging cases, particularly with rare, unusual, and emerging diseases that have not yet been described in reference texts and only infrequently reported in the medical literature.

Disclosures

Jonathan S. Zipursky, Keri T. Holmes-Maybank, Steven L. Shumak, and Ashley A. Ducketthave none to declare.

A 22-year-old man presented to a Canadian community hospital emergency department complaining of 2-3 weeks of abdominal pain and bloating associated with early satiety. He also noted weight loss of 20 pounds over the preceding months, leg and abdominal swelling with increased girth, and 1-2 loose, nonbloody stools per day.

Early satiety and bloating are nonspecific symptoms that can be due to gastroesophageal reflux disease, peptic ulcer disease, gastrointestinal obstruction, or gastroparesis. Weight loss in a young person, particularly if >5% of body weight, is concerning for a serious underlying medical issue. It could reflect reduced intake due to anorexia, odynophagia, or dysphagia or increased energy expenditure due to an inflammatory state such as infection or rheumatic disease. The etiology of the swelling needs to be elucidated. It may be due to increased hydrostatic forces as in heart failure, venous or lymphatic obstruction, or from lowered oncotic pressure resulting from hepatic disease, nephrotic syndrome, severe malnutrition (nonbloody loose stools), or a protein losing enteropathy.

The patient was transferred to a tertiary care center for closer access to specialty consultation. He described generalized abdominal pain increasing in intensity over three weeks; bilateral lower extremity, scrotal, abdominal wall, and sacral edema; and mild dyspnea on exertion. The early satiety was not associated with dysphagia, odynophagia, nausea, or vomiting. He denied fevers, chills, night sweats, nausea, vomiting, jaundice, easy bruising, orthopnea, paroxysmal nocturnal dyspnea (PND), or chest pain. His past medical history included asthma treated with fluticasone/salmeterol and albuterol. He was a Canadian of East Asian descent working as a plumber. He previously smoked three to four cigarettes per day for six years. He stopped smoking one month before presentation. He had one alcoholic beverage per week and smoked marijuana weekly. He denied any family history of similar symptoms or malignancy.The differential diagnosis for weight loss and anasarca is broad and includes malignancies, infectious diseases, rheumatic or inflammatory disorders, malabsorption, and advanced cardiac, renal, or liver disease. His history does not classically point in one direction. The mild dyspnea on exertion may be due to cardiac disease, but it is unlikely in the absence of orthopnea and PND. The dyspnea could be due to increased abdominal pressure if ascites are present, his underlying asthma, or another etiology such as anemia. Fevers, chills, and/or night sweats can be expected in infections and some malignancies, but their absence does not exclude infections and malignancies from the differential diagnoses. Particular attention should be paid to lymphadenopathy on the physical examination. The presence of an umbilical nodule (Sister Mary Joseph sign) could indicate a malignancy (gastrointestinal or lymphoma).

The differential diagnosis for weight loss and anasarca is broad and includes malignancies, infectious diseases, rheumatic or inflammatory disorders, malabsorption, and advanced cardiac, renal, or liver disease. His history does not classically point in one direction. The mild dyspnea on exertion may be due to cardiac disease, but it is unlikely in the absence of orthopnea and PND. The dyspnea could be due to increased abdominal pressure if ascites are present, his underlying asthma, or another etiology such as anemia. Fevers, chills, and/or night sweats can be expected in infections and some malignancies, but their absence does not exclude infections and malignancies from the differential diagnoses. Particular attention should be paid to lymphadenopathy on the physical examination. The presence of an umbilical nodule (Sister Mary Joseph sign) could indicate a malignancy (gastrointestinal or lymphoma).

On physical examination, his temperature was 38.1°C, heart rate was 138 beats per minute, blood pressure was 123/86 mm Hg, respiratory rate was 20 breaths per minute, and oxygen saturation was 97% on room air. He appeared uncomfortable and diaphoretic. No scleral icterus or jaundice was appreciated. There were no palpable cervical, axillary, or inguinal lymph nodes. Cardiac examination revealed tachycardia and no murmurs, rubs, gallops, or jugular venous distention. Abdominal examination revealed abdominal distention, diffuse tenderness to deep palpation, bulging flanks, and a positive fluid wave. Liver and spleen could not be palpated or percussed secondary to abdominal distention. He had pitting bilateral lower extremity edema that extended to and included the scrotum. Neurologic and pulmonary examinations were unremarkable.

His examination reveals low-grade fever, tachycardia, and diaphoresis. Whether this represents progression of his primary disease or he has acutely developed a superimposed infection is uncertain at this point. He has notable anasarca but no jugular venous distention, crackles, or S3 gallop. The lack of evidence of pulmonary edema or increased central venous pressure on physical examination increases the likelihood of cirrhosis, hypoalbuminemia, or obstruction (lymphatic or venous) and decreases the likelihood of heart failure as the etiology of his peripheral edema and likely ascites. Despite the prominence of gastrointestinal symptoms, he has neither jaundice nor stigmata of chronic liver disease. Periorbital edema, which may be present in nephrotic syndrome, is also absent. Although he has no palpable peripheral lymphadenopathy, malignancy remains a concern.

Testing should include urinalysis for proteinuria and coagulation studies to assess synthetic function of the liver. Abdominal ultrasound is indicated to confirm ascites. If present, a diagnostic paracentesis should be performed to rule out spontaneous bacterial peritonitis and determine whether the ascites is from portal hypertension, hypoalbuminemia, or peritoneal disease. If the transaminases are elevated or if the ascitic fluid is concerning for malignancy, he will need a computed tomography (CT) of the abdomen and pelvis. A protein losing enteropathy due to malignancies (gastric cancer or lymphoma), rheumatic disease (systemic lupus erythematosus [SLE]), or infiltrative disease (amyloid) is also a possibility. If the other studies are unrevealing, stool should be sent for alpha-1 antitrypsin.

Laboratory studies revealed hemoglobin 7.8 g/dL, platelets 53 k/mm³, white blood cell count (WBC) 10.6 k/mm³, alkaline phosphatase (ALP) 217 U/L, albumin 2.7 g/dL, reticulocyte count 3 k/mm³ (reference range, 30-110 k/mm³), and ferritin 1,310 ng/mL (reference range, 20-400 ng/L). Serum aminotransferase levels, bilirubin, coagulation panel, electrolytes, and creatinine were normal. Urinalysis was negative for blood, leukocytes, and protein. Diagnostic paracentesis demonstrated a serum-ascites-albumin gradient (SAAG) of two and macrophage predominance (WBC 250 U/L). Ascitic fluid cytology and culture were negative. Blood cultures, human immunodeficiency virus (HIV)-1 and 2, cytomegalovirus (CMV), and Epstein–Barr virus (EBV) serologies were negative. Viral serologies for hepatitis A, B, and C were negative. Antinuclear antibody (ANA), anti-ds DNA, antineutrophilic cytoplasmic antibody (ANCA), serum angiotensin-converting enzyme (ACE) level, and quantitative immunoglobulin levels were all within the normal range. Chest, abdomen, and pelvis CT with contrast revealed large-volume abdominal and pelvic ascites, diffuse subcutaneous edema (Figure 1), modest hepatosplenomegaly, small bilateral pleural effusions, and mediastinal, axillary, mesenteric, periportal, peripancreatic, and retroperitoneal lymphadenopathy (Figure 2).

Some of the more common viral etiologies of generalized lymphadenopathy and cytopenias are unlikely because serologies for HIV, hepatitis B and C, EBV, and CMV are negative. Tuberculosis fits with the insidious nature of his presentation and remains on the differential although a low SAAG would be expected. From a rheumatologic standpoint, the lack of characteristic findings on history and physical examination and the negative ANA and anti-ds DNA results make SLE unlikely. Although elevated in the majority of untreated sarcoid patients, a normal ACE level is not sufficient to rule out this diagnosis. IgG, IgA, and IgM levels would be low if there was significant gastrointestinal protein loss and elevated in MCD. The markedly increased ferritin level, an acute-phase reactant often elevated in the setting of inflammation or malignancy, raises suspicion for adult Still’s disease (despite the lack of characteristic arthralgias and/or rash) and hemophagocytic lymphohistiocytosis (HLH).

A SAAG greater than or equal to 1.1 indicates the presence of portal hypertension. Portal hypertension most often results from cirrhosis for which this patient has no apparent clinical findings. Etiologies of noncirrhotic portal hypertension are classified as prehepatic, intrahepatic, and posthepatic. There is no clinical or radiologic evidence of portal or splenic vein thrombosis (prehepatic) or heart failure (posthepatic). Possible intrahepatic etiologies include malignancy and sarcoid. Although uncommon, patients with malignancy-related ascites may have a high SAAG without coexisting cirrhosis. This occurs if there is portal hypertension due to extensive metastases in the liver or involvement of the portal venous system. The cytology of the ascitic fluid is negative. However, cytology is <80% sensitive in the absence of peritoneal carcinomatosis.

Hematology was consulted for evaluation of the lymphadenopathy, anemia, and thrombocytopenia and recommended bone marrow and excisional lymph node biopsies. Bone marrow biopsy showed trilineage hypercellularity (Figure 3A) with reduced erythropoiesis and reticulin fibrosis (Figure 3B). An axillary lymph node biopsy with flow cytometry was nondiagnostic for a lymphoproliferative disorder or malignancy.

Five days after discharge, he was readmitted with worsening anasarca, massive ascites, and acute kidney injury. Admission laboratory studies revealed creatinine 1.66 mg/dL, hemoglobin 11.5 g/dL, and platelets 94 k/mm³. In addition, his ferritin level was 1,907 ng/L (reference range, 20-400 ng/L), erythrocyte sedimentation rate (ESR) was 50 mm/h (reference range, 0-20 mm/h), and C-reactive protein concentration (CRP) was 12.1 mg/dL (reference range, 0-0.5 mg/dL).

Steroids are used to treat a wide variety of illnesses, some of which are still under consideration in this patient including lymphoma, MCD, adult Still’s disease, and HLH. His symptoms recurred quickly after discontinuation of steroids in the setting of elevated ferritin, ESR, and CRP levels reflecting marked ongoing inflammation. Serologic testing for soluble IL-2 receptor, often elevated in MCD and HLH, should be performed. Excisional biopsy of an accessible node should be performed urgently.

His acute kidney injury resolved; however, he continued to have intermittent fevers, anemia, thromobocytopenia, lymphadenopathy, and hepatosplenomegaly. A hematology case-conference recommended testing for HLH, including soluble IL-2 receptor (CD25), soluble CD163, and natural killer cell degranulation assay, all of which were negative. A right inguinal lymph node biopsy revealed reactive lymphoid tissue and stained negative for HHV-8. Based on the lack of an alternative diagnosis (particularly lymphoma), the presence of multiple areas of lymphadenopathy, anemia, fevers, organomegaly, weight loss, reactive lymphoid tissue on lymph node biopsy, and elevated CRP and ESR, a working diagnosis of MCD was made. The negative HHV-8 testing was consistent with idiopathic MCD (iMCD); however, features inconsistent with iMCD included lack of polyclonal hypergammaglobulinemia and the presence of significant anasarca and thrombocytopenia. Therefore, an internet search was performed using the patient’s salient symptoms and findings. The search revealed a few recently published case reports of a rare variant of iMCD, TAFRO syndrome. TAFRO syndrome, characterized by thrombocytopenia, anasarca, fever, reticulin fibrosis and/or renal insufficiency, and organomegaly, fully explained the patient’s presentation. He was started on prednisone, rituximab (anti-CD20 antibody), and furosemide. After one month of treatment, he showed complete resolution of cytopenias, lymphadenopathy, organomegaly, anasarca, and ascites. Therapy continued for approximately three months, and he has remained symptom-free.

Castleman’s disease (CD) is a rare lymphoproliferative disorder divided into unicentric (solitary enlarged lymph node) and multicentric (multifocal enlarged lymph nodes).¹ MCD typically presents with systemic inflammation, reactive proliferation of benign lymphocytes, multifocal lymphadenopathy, elevated inflammatory markers, anemia, hypoalbuminemia, and polyclonal gammaglobulinemia.¹ It is hypothesized that HHV-8 drives the systemic inflammation of MCD via high levels of interleukin-6 (IL-6) activity.¹ iMCD is an HHV-8-negative variant of MCD.¹

TAFRO syndrome was first described in 2010 in three Japanese patients demonstrating high fever, anasarca, hepatosplenomegaly, lymphadenopathy, severe thrombocytopenia, and reticulin fibrosis.² In 2015, the All Japan TAFRO Syndrome Research Group recognized TAFRO syndrome as a variant of iMCD and created diagnostic criteria and a severity classification system.³ Major criteria consist of anasarca, including pleural effusion and/or ascites identified on CT scan and general edema, thrombocytopenia (platelet count <100 k/mm³), and systemic inflammation (fever >37.5°C and/or serum CRP greater than or equal to 2 mg/dL).³ Two of four minor criteria must be met, which include (1) lymph node histology consistent with CD, (2) reticulin myelofibrosis and/or increased number of megakaryocytes in bone marrow, (3) mild organomegaly, including hepatomegaly, splenomegaly, and lymphadenopathy <1.5 cm in diameter identified on CT scan, and (4) progressive renal insufficiency (serum creatinine >1.2 mg/dL in males or >1.0 mg/dL in females).³ In addition, several patients with TAFRO syndrome demonstrate elevated ALP, low-normal LDH, elevated vascular endothelial growth factor, elevated IL-6, microcytic anemia, and slight polyclonal hypergammopathy.³ Malignancies such as lymphoma and myeloma, autoimmune diseases such as SLE and ANCA-associated vasculitis, infectious diseases such as those caused by mycobacteria, and POEMS (polyneuropathy, organomegaly, endocrine diseases, M-protein, and skin lesions) syndrome must be excluded to diagnose TAFRO syndrome.^3,4

The pathophysiology of TAFRO syndrome is unknown, and it is unclear whether the syndrome is truly a variant of iMCD or a distinct entity.³ IL-6 is typically only mildly elevated in TAFRO syndrome, without the consequent thrombocytosis and polyclonal hypergammaglobulinemia seen in MCD, which is associated with higher levels of IL-6.¹ Multiple non-HHV-8 mechanisms for TAFRO syndrome have been proposed, including (1) systemic inflammation, autoimmune/autoinflammatory mechanisms, (2) neoplastic, ectopic cytokine secretion by malignant or benign tumor cells, and/or (3) infectious, such as non-HHV-8 virus.⁵

Immunosuppression is the mainstay of treatment for TAFRO syndrome based on recommendations from the 2015 TAFRO Research Group.³ Glucocorticoids are considered first-line therapy.³ Cyclosporin A is recommended for individuals refractory to glucocorticoids.³ In patients with a contraindication to cyclosporin A, anti-IL-6 receptor antibodies such as tocilizumab (approved for treatment of iMCD in Japan) and siltuximab (approved for treatment of iMCD in North America and Europe) or the anti-CD20 antibody rituximab should be prescribed.³ There is evidence for the thrombopoietin receptor agonists romiplostim and eltrombopag to treat persistent thrombocytopenia.³ Additional treatments for refractory TAFRO syndrome include IVIG and plasma exchange, chemotherapy (cyclophosphamide, doxorubicin, vincristine, prednisolone), and thalidomide.^3,6

Little is known about the epidemiologic characterization of TAFRO syndrome as less than 40 cases of TAFRO syndrome have been reported in the United States, Asia, and Europe.

• Key clinical and pathologic features of TAFRO syndrome include thrombocytopenia, anasarca, fever, reticulin fibrosis and/or renal insufficiency, and organomegaly.
• TAFRO syndrome may be under-recognized due to very recent characterization and no previous ICD code for CD.
• TAFRO syndrome experts recommend immunosuppression for treatment of TAFRO syndrome, including glucocorticoids as first-line treatment.
• Internet searches can be helpful in the diagnosis of challenging cases, particularly with rare, unusual, and emerging diseases that have not yet been described in reference texts and only infrequently reported in the medical literature.

Disclosures

Jonathan S. Zipursky, Keri T. Holmes-Maybank, Steven L. Shumak, and Ashley A. Ducketthave none to declare.

A 22-year-old man presented to a Canadian community hospital emergency department complaining of 2-3 weeks of abdominal pain and bloating associated with early satiety. He also noted weight loss of 20 pounds over the preceding months, leg and abdominal swelling with increased girth, and 1-2 loose, nonbloody stools per day.

Early satiety and bloating are nonspecific symptoms that can be due to gastroesophageal reflux disease, peptic ulcer disease, gastrointestinal obstruction, or gastroparesis. Weight loss in a young person, particularly if >5% of body weight, is concerning for a serious underlying medical issue. It could reflect reduced intake due to anorexia, odynophagia, or dysphagia or increased energy expenditure due to an inflammatory state such as infection or rheumatic disease. The etiology of the swelling needs to be elucidated. It may be due to increased hydrostatic forces as in heart failure, venous or lymphatic obstruction, or from lowered oncotic pressure resulting from hepatic disease, nephrotic syndrome, severe malnutrition (nonbloody loose stools), or a protein losing enteropathy.

The patient was transferred to a tertiary care center for closer access to specialty consultation. He described generalized abdominal pain increasing in intensity over three weeks; bilateral lower extremity, scrotal, abdominal wall, and sacral edema; and mild dyspnea on exertion. The early satiety was not associated with dysphagia, odynophagia, nausea, or vomiting. He denied fevers, chills, night sweats, nausea, vomiting, jaundice, easy bruising, orthopnea, paroxysmal nocturnal dyspnea (PND), or chest pain. His past medical history included asthma treated with fluticasone/salmeterol and albuterol. He was a Canadian of East Asian descent working as a plumber. He previously smoked three to four cigarettes per day for six years. He stopped smoking one month before presentation. He had one alcoholic beverage per week and smoked marijuana weekly. He denied any family history of similar symptoms or malignancy.The differential diagnosis for weight loss and anasarca is broad and includes malignancies, infectious diseases, rheumatic or inflammatory disorders, malabsorption, and advanced cardiac, renal, or liver disease. His history does not classically point in one direction. The mild dyspnea on exertion may be due to cardiac disease, but it is unlikely in the absence of orthopnea and PND. The dyspnea could be due to increased abdominal pressure if ascites are present, his underlying asthma, or another etiology such as anemia. Fevers, chills, and/or night sweats can be expected in infections and some malignancies, but their absence does not exclude infections and malignancies from the differential diagnoses. Particular attention should be paid to lymphadenopathy on the physical examination. The presence of an umbilical nodule (Sister Mary Joseph sign) could indicate a malignancy (gastrointestinal or lymphoma).

The differential diagnosis for weight loss and anasarca is broad and includes malignancies, infectious diseases, rheumatic or inflammatory disorders, malabsorption, and advanced cardiac, renal, or liver disease. His history does not classically point in one direction. The mild dyspnea on exertion may be due to cardiac disease, but it is unlikely in the absence of orthopnea and PND. The dyspnea could be due to increased abdominal pressure if ascites are present, his underlying asthma, or another etiology such as anemia. Fevers, chills, and/or night sweats can be expected in infections and some malignancies, but their absence does not exclude infections and malignancies from the differential diagnoses. Particular attention should be paid to lymphadenopathy on the physical examination. The presence of an umbilical nodule (Sister Mary Joseph sign) could indicate a malignancy (gastrointestinal or lymphoma).

On physical examination, his temperature was 38.1°C, heart rate was 138 beats per minute, blood pressure was 123/86 mm Hg, respiratory rate was 20 breaths per minute, and oxygen saturation was 97% on room air. He appeared uncomfortable and diaphoretic. No scleral icterus or jaundice was appreciated. There were no palpable cervical, axillary, or inguinal lymph nodes. Cardiac examination revealed tachycardia and no murmurs, rubs, gallops, or jugular venous distention. Abdominal examination revealed abdominal distention, diffuse tenderness to deep palpation, bulging flanks, and a positive fluid wave. Liver and spleen could not be palpated or percussed secondary to abdominal distention. He had pitting bilateral lower extremity edema that extended to and included the scrotum. Neurologic and pulmonary examinations were unremarkable.

His examination reveals low-grade fever, tachycardia, and diaphoresis. Whether this represents progression of his primary disease or he has acutely developed a superimposed infection is uncertain at this point. He has notable anasarca but no jugular venous distention, crackles, or S3 gallop. The lack of evidence of pulmonary edema or increased central venous pressure on physical examination increases the likelihood of cirrhosis, hypoalbuminemia, or obstruction (lymphatic or venous) and decreases the likelihood of heart failure as the etiology of his peripheral edema and likely ascites. Despite the prominence of gastrointestinal symptoms, he has neither jaundice nor stigmata of chronic liver disease. Periorbital edema, which may be present in nephrotic syndrome, is also absent. Although he has no palpable peripheral lymphadenopathy, malignancy remains a concern.

Testing should include urinalysis for proteinuria and coagulation studies to assess synthetic function of the liver. Abdominal ultrasound is indicated to confirm ascites. If present, a diagnostic paracentesis should be performed to rule out spontaneous bacterial peritonitis and determine whether the ascites is from portal hypertension, hypoalbuminemia, or peritoneal disease. If the transaminases are elevated or if the ascitic fluid is concerning for malignancy, he will need a computed tomography (CT) of the abdomen and pelvis. A protein losing enteropathy due to malignancies (gastric cancer or lymphoma), rheumatic disease (systemic lupus erythematosus [SLE]), or infiltrative disease (amyloid) is also a possibility. If the other studies are unrevealing, stool should be sent for alpha-1 antitrypsin.

Laboratory studies revealed hemoglobin 7.8 g/dL, platelets 53 k/mm³, white blood cell count (WBC) 10.6 k/mm³, alkaline phosphatase (ALP) 217 U/L, albumin 2.7 g/dL, reticulocyte count 3 k/mm³ (reference range, 30-110 k/mm³), and ferritin 1,310 ng/mL (reference range, 20-400 ng/L). Serum aminotransferase levels, bilirubin, coagulation panel, electrolytes, and creatinine were normal. Urinalysis was negative for blood, leukocytes, and protein. Diagnostic paracentesis demonstrated a serum-ascites-albumin gradient (SAAG) of two and macrophage predominance (WBC 250 U/L). Ascitic fluid cytology and culture were negative. Blood cultures, human immunodeficiency virus (HIV)-1 and 2, cytomegalovirus (CMV), and Epstein–Barr virus (EBV) serologies were negative. Viral serologies for hepatitis A, B, and C were negative. Antinuclear antibody (ANA), anti-ds DNA, antineutrophilic cytoplasmic antibody (ANCA), serum angiotensin-converting enzyme (ACE) level, and quantitative immunoglobulin levels were all within the normal range. Chest, abdomen, and pelvis CT with contrast revealed large-volume abdominal and pelvic ascites, diffuse subcutaneous edema (Figure 1), modest hepatosplenomegaly, small bilateral pleural effusions, and mediastinal, axillary, mesenteric, periportal, peripancreatic, and retroperitoneal lymphadenopathy (Figure 2).

Some of the more common viral etiologies of generalized lymphadenopathy and cytopenias are unlikely because serologies for HIV, hepatitis B and C, EBV, and CMV are negative. Tuberculosis fits with the insidious nature of his presentation and remains on the differential although a low SAAG would be expected. From a rheumatologic standpoint, the lack of characteristic findings on history and physical examination and the negative ANA and anti-ds DNA results make SLE unlikely. Although elevated in the majority of untreated sarcoid patients, a normal ACE level is not sufficient to rule out this diagnosis. IgG, IgA, and IgM levels would be low if there was significant gastrointestinal protein loss and elevated in MCD. The markedly increased ferritin level, an acute-phase reactant often elevated in the setting of inflammation or malignancy, raises suspicion for adult Still’s disease (despite the lack of characteristic arthralgias and/or rash) and hemophagocytic lymphohistiocytosis (HLH).

A SAAG greater than or equal to 1.1 indicates the presence of portal hypertension. Portal hypertension most often results from cirrhosis for which this patient has no apparent clinical findings. Etiologies of noncirrhotic portal hypertension are classified as prehepatic, intrahepatic, and posthepatic. There is no clinical or radiologic evidence of portal or splenic vein thrombosis (prehepatic) or heart failure (posthepatic). Possible intrahepatic etiologies include malignancy and sarcoid. Although uncommon, patients with malignancy-related ascites may have a high SAAG without coexisting cirrhosis. This occurs if there is portal hypertension due to extensive metastases in the liver or involvement of the portal venous system. The cytology of the ascitic fluid is negative. However, cytology is <80% sensitive in the absence of peritoneal carcinomatosis.

Hematology was consulted for evaluation of the lymphadenopathy, anemia, and thrombocytopenia and recommended bone marrow and excisional lymph node biopsies. Bone marrow biopsy showed trilineage hypercellularity (Figure 3A) with reduced erythropoiesis and reticulin fibrosis (Figure 3B). An axillary lymph node biopsy with flow cytometry was nondiagnostic for a lymphoproliferative disorder or malignancy.

Five days after discharge, he was readmitted with worsening anasarca, massive ascites, and acute kidney injury. Admission laboratory studies revealed creatinine 1.66 mg/dL, hemoglobin 11.5 g/dL, and platelets 94 k/mm³. In addition, his ferritin level was 1,907 ng/L (reference range, 20-400 ng/L), erythrocyte sedimentation rate (ESR) was 50 mm/h (reference range, 0-20 mm/h), and C-reactive protein concentration (CRP) was 12.1 mg/dL (reference range, 0-0.5 mg/dL).

Steroids are used to treat a wide variety of illnesses, some of which are still under consideration in this patient including lymphoma, MCD, adult Still’s disease, and HLH. His symptoms recurred quickly after discontinuation of steroids in the setting of elevated ferritin, ESR, and CRP levels reflecting marked ongoing inflammation. Serologic testing for soluble IL-2 receptor, often elevated in MCD and HLH, should be performed. Excisional biopsy of an accessible node should be performed urgently.

His acute kidney injury resolved; however, he continued to have intermittent fevers, anemia, thromobocytopenia, lymphadenopathy, and hepatosplenomegaly. A hematology case-conference recommended testing for HLH, including soluble IL-2 receptor (CD25), soluble CD163, and natural killer cell degranulation assay, all of which were negative. A right inguinal lymph node biopsy revealed reactive lymphoid tissue and stained negative for HHV-8. Based on the lack of an alternative diagnosis (particularly lymphoma), the presence of multiple areas of lymphadenopathy, anemia, fevers, organomegaly, weight loss, reactive lymphoid tissue on lymph node biopsy, and elevated CRP and ESR, a working diagnosis of MCD was made. The negative HHV-8 testing was consistent with idiopathic MCD (iMCD); however, features inconsistent with iMCD included lack of polyclonal hypergammaglobulinemia and the presence of significant anasarca and thrombocytopenia. Therefore, an internet search was performed using the patient’s salient symptoms and findings. The search revealed a few recently published case reports of a rare variant of iMCD, TAFRO syndrome. TAFRO syndrome, characterized by thrombocytopenia, anasarca, fever, reticulin fibrosis and/or renal insufficiency, and organomegaly, fully explained the patient’s presentation. He was started on prednisone, rituximab (anti-CD20 antibody), and furosemide. After one month of treatment, he showed complete resolution of cytopenias, lymphadenopathy, organomegaly, anasarca, and ascites. Therapy continued for approximately three months, and he has remained symptom-free.

Castleman’s disease (CD) is a rare lymphoproliferative disorder divided into unicentric (solitary enlarged lymph node) and multicentric (multifocal enlarged lymph nodes).¹ MCD typically presents with systemic inflammation, reactive proliferation of benign lymphocytes, multifocal lymphadenopathy, elevated inflammatory markers, anemia, hypoalbuminemia, and polyclonal gammaglobulinemia.¹ It is hypothesized that HHV-8 drives the systemic inflammation of MCD via high levels of interleukin-6 (IL-6) activity.¹ iMCD is an HHV-8-negative variant of MCD.¹

TAFRO syndrome was first described in 2010 in three Japanese patients demonstrating high fever, anasarca, hepatosplenomegaly, lymphadenopathy, severe thrombocytopenia, and reticulin fibrosis.² In 2015, the All Japan TAFRO Syndrome Research Group recognized TAFRO syndrome as a variant of iMCD and created diagnostic criteria and a severity classification system.³ Major criteria consist of anasarca, including pleural effusion and/or ascites identified on CT scan and general edema, thrombocytopenia (platelet count <100 k/mm³), and systemic inflammation (fever >37.5°C and/or serum CRP greater than or equal to 2 mg/dL).³ Two of four minor criteria must be met, which include (1) lymph node histology consistent with CD, (2) reticulin myelofibrosis and/or increased number of megakaryocytes in bone marrow, (3) mild organomegaly, including hepatomegaly, splenomegaly, and lymphadenopathy <1.5 cm in diameter identified on CT scan, and (4) progressive renal insufficiency (serum creatinine >1.2 mg/dL in males or >1.0 mg/dL in females).³ In addition, several patients with TAFRO syndrome demonstrate elevated ALP, low-normal LDH, elevated vascular endothelial growth factor, elevated IL-6, microcytic anemia, and slight polyclonal hypergammopathy.³ Malignancies such as lymphoma and myeloma, autoimmune diseases such as SLE and ANCA-associated vasculitis, infectious diseases such as those caused by mycobacteria, and POEMS (polyneuropathy, organomegaly, endocrine diseases, M-protein, and skin lesions) syndrome must be excluded to diagnose TAFRO syndrome.^3,4

The pathophysiology of TAFRO syndrome is unknown, and it is unclear whether the syndrome is truly a variant of iMCD or a distinct entity.³ IL-6 is typically only mildly elevated in TAFRO syndrome, without the consequent thrombocytosis and polyclonal hypergammaglobulinemia seen in MCD, which is associated with higher levels of IL-6.¹ Multiple non-HHV-8 mechanisms for TAFRO syndrome have been proposed, including (1) systemic inflammation, autoimmune/autoinflammatory mechanisms, (2) neoplastic, ectopic cytokine secretion by malignant or benign tumor cells, and/or (3) infectious, such as non-HHV-8 virus.⁵

Immunosuppression is the mainstay of treatment for TAFRO syndrome based on recommendations from the 2015 TAFRO Research Group.³ Glucocorticoids are considered first-line therapy.³ Cyclosporin A is recommended for individuals refractory to glucocorticoids.³ In patients with a contraindication to cyclosporin A, anti-IL-6 receptor antibodies such as tocilizumab (approved for treatment of iMCD in Japan) and siltuximab (approved for treatment of iMCD in North America and Europe) or the anti-CD20 antibody rituximab should be prescribed.³ There is evidence for the thrombopoietin receptor agonists romiplostim and eltrombopag to treat persistent thrombocytopenia.³ Additional treatments for refractory TAFRO syndrome include IVIG and plasma exchange, chemotherapy (cyclophosphamide, doxorubicin, vincristine, prednisolone), and thalidomide.^3,6

Little is known about the epidemiologic characterization of TAFRO syndrome as less than 40 cases of TAFRO syndrome have been reported in the United States, Asia, and Europe.

• Key clinical and pathologic features of TAFRO syndrome include thrombocytopenia, anasarca, fever, reticulin fibrosis and/or renal insufficiency, and organomegaly.
• TAFRO syndrome may be under-recognized due to very recent characterization and no previous ICD code for CD.
• TAFRO syndrome experts recommend immunosuppression for treatment of TAFRO syndrome, including glucocorticoids as first-line treatment.
• Internet searches can be helpful in the diagnosis of challenging cases, particularly with rare, unusual, and emerging diseases that have not yet been described in reference texts and only infrequently reported in the medical literature.

Disclosures

Jonathan S. Zipursky, Keri T. Holmes-Maybank, Steven L. Shumak, and Ashley A. Ducketthave none to declare.

Alarms from bedside continuous physiologic monitors (CPMs) occur frequently in children’s hospitals and can lead to harm. Recent studies conducted in children’s hospitals have identified alarm rates of up to 152 alarms per patient per day outside of the intensive care unit,^1-3 with as few as 1% of alarms being considered clinically important.⁴ Excessive alarms have been linked to alarm fatigue, when providers become desensitized to and may miss alarms indicating impending patient deterioration. Alarm fatigue has been identified by national patient safety organizations as a patient safety concern given the risk of patient harm.^5-7 Despite these concerns, CPMs are routinely used: up to 48% of pediatric patients in nonintensive care units at children’s hospitals are monitored.²

Although the low number of alarms that receive responses has been well-described,^8,9 the reasons why clinicians do or do not respond to alarms are unclear. A study conducted in an adult perioperative unit noted prolonged nurse response times for patients with high alarm rates.¹⁰ A second study conducted in the pediatric inpatient setting demonstrated a dose-response effect and noted progressively prolonged nurse response times with increased rates of nonactionable alarms.^4,11 Findings from another study suggested that underlying factors are highly complex and may be a result of excessive alarms, clinician characteristics, and working conditions (eg, workload and unit noise level).¹² Evidence also suggests that humans have difficulty distinguishing the importance of alarms in situations where multiple alarm tones are used, a common scenario in hospitals.^13,14 Understanding the factors that contribute to clinicians responding or not responding to CPM alarms will be crucial for addressing this serious patient safety issue.

An enhanced understanding of why nurses respond to alarms in daily practice will inform intervention development and improvement work. In the long term, this information could help improve systems for monitoring pediatric inpatients that are less prone to issues with alarm fatigue. The objective of this qualitative study, which employed structured observation, was to describe how bedside nurses think about and act upon bedside monitor alarms in a general pediatric inpatient unit.

Study Design and Setting

This prospective observational study took place on a 48-bed hospital medicine unit at a large, freestanding children’s hospital with >650 beds and >19,000 annual admissions. General Electric (Little Chalfont, United Kingdom) physiologic monitors (models Dash 3000, 4000, and 5000) were used at the time of the study, and nurses could be notified of monitor alarms in four ways: First, an in-room auditory alarm sounds. Second, a light positioned above the door outside of each patient room blinks for alarms that are at a “warning” or “critical level” (eg ventricular tachycardia or low oxygen saturation). Third, audible alarms occur at the unit’s central monitoring station. Lastly, another staff member can notify the patient’s nurse via in-person conversion or secure smart phone communication. On the study unit, CPMs are initiated and discontinued through a physician order.

This study was reviewed and approved by the hospital’s institutional review board.

Study Population

We used a purposive recruitment strategy to enroll bedside nurses working on general hospital medicine units, stratified to ensure varying levels of experience and primary shifts (eg, day vs night). We planned to conduct approximately two observations with each participating nurse and to continue collecting data until we could no longer identify new insights in terms of responses to alarms (ie, thematic saturation¹⁵). Observations were targeted to cover times of day that coincided with increased rates of distraction. These times included just prior to and after the morning and evening change of shifts (7:00 am and 7:00 pm), during morning rounds (8:00 am-12:00 pm), and heavy admission times (12:00 pm-10:00 pm). After written informed consent, a nurse was eligible for observation during his/her shift if he/she was caring for at least one monitored patient. Enrolled nurses were made aware of the general study topic but were blinded to the study team’s hypotheses.

Data Sources

Prior to data collection, the research team, which consisted of physicians, bedside nurses, research coordinators, and a human factors expert, created a system for categorizing alarm responses. Categories for observed responses were based on the location and corresponding action taken. Initial categories were developed a priori from existing literature and expanded through input from the multidisciplinary study team, then vetted with bedside staff, and finally pilot tested through >4 hours of observations, thus producing the final categories. These categories were entered into a work-sampling program (WorkStudy by Quetech Ltd., Waterloo, Ontario, Canada) to facilitate quick data recording during observations.

The hospital uses a central alarm collection software (BedMasterEx by Anandic Medical Systems, Feuerthalen, Switzerland), which permitted the collection of date, time, trigger (eg, high heart rate), and level (eg, crisis, warning) of the generated CPM alarms. Alarms collected are based on thresholds preset at the bedside monitor. The central collection software does not differentiate between accurate (eg, correctly representing the physiologic state of the patient) and inaccurate alarms.

Observation Procedure

At the time of observation, nurse demographic information (eg, primary shift worked and years working as a nurse) was obtained. A brief preobservation questionnaire was administered to collect patient information (eg, age and diagnosis) and the nurses’ perspectives on the necessity of monitors for each monitored patient in his/her care.

The observer shadowed the nurse for a two-hour block of his/her shift. During this time, nurses were instructed to “think aloud” as they responded to alarms (eg, “I notice the oxygen saturation monitor alarming off, but the probe has fallen off”). A trained observer (AML or KMT) recorded responses verbalized by the nurse and his/her reaction by selecting the appropriate category using the work-sampling software. Data were also collected on the vital sign associated with the alarm (eg, heart rate). Moreover, the observer kept written notes to provide context for electronically recorded data. Alarms that were not verbalized by the nurse were not counted. Similarly, alarms that were noted outside of the room by the nurse were not classified by vital sign unless the nurse confirmed with the bedside monitor. Observers did not adjudicate the accuracy of the alarms. The session was stopped if monitors were discontinued during the observation period. Alarm data generated by the bedside monitor were pulled for each patient room after observations were completed.

Analysis

Descriptive statistics were used to assess the percentage of each nurse response category and each alarm type (eg, heart rate and respiratory rate). The observed alarm rate was calculated by taking the total number of observed alarms (ie, alarms noted by the nurse) divided by the total number of patient-hours observed. The monitor-generated alarm rate was calculated by taking the total number of alarms from the bedside-alarm generated data divided by the number of patient-hours observed.

Electronically recorded observations using the work-sampling program were cross-referenced with hand-written field notes to assess for any discrepancies or identify relevant events not captured by the program. Three study team members (AML, KMT, and ACS) reviewed each observation independently and compared field notes to ensure accurate categorization. Discrepancies were referred to the larger study group in cases of uncertainty.

Nine nurses had monitored patients during the available observations and participated in 19 observation sessions, which included 35 monitored patients for a total of 61.3 patient-hours of observation. Nurses were observed for a median of two times each (range 1-4). The median number of monitored patients during a single observation session was two (range 1-3). Observed nurses were female with a median of eight years of experience (range 0.5-26 years). Patients represented a broad range of age categories and were hospitalized with a variety of diagnoses (Table). Nurses, when queried at the start of the observation, felt that monitors were necessary for 29 (82.9%) of the observed patients given either patient condition or unit policy.

A total of 207 observed nurse responses to alarms occurred during the study period for a rate of 3.4 responses per patient per hour. Of the total number of responses, 45 (21.7%) were noted outside of a patient room, and in 15 (33.3%) the nurse chose to go to the room. The other 162 were recorded when the nurse was present in the room when the alarm activated. Of the 177 in-person nurse responses, 50 were related to a pulse oximetry alarm, 66 were related to a heart rate alarm, and 61 were related to a respiratory rate alarm. The most common observed in-person response to an alarm involved the nurse judging that no intervention was necessary (n = 152, 73.1%). Only 14 (7% of total responses) observed in-person responses involved a clinical intervention, such as suctioning or titrating supplemental oxygen. Findings are summarized in the Figure and describe nurse-verbalized reasons to further assess (or not) and then whether the nurse chose to take action (or not) after an alarm.

We characterized responses to physiologic monitor alarms by a group of nurses with a range of experience levels. We found that most nurse responses to alarms in continuously monitored general pediatric patients involved no intervention, and further assessment was often not conducted for alarms that occurred outside of the room if the nurse noted otherwise reassuring clinical context. Observed responses occurred for 36% of alarms during the study period when compared with bedside monitor-alarm generated data. Overall, only 14 clinical interventions were noted among the observed responses. Nurses noted that they felt the monitors were necessary for 82.9% of monitored patients because of the clinical context or because of unit policy.

Our study findings highlight some potential contradictions in the current widespread use of CPMs in general pediatric units and how clinicians respond to them in practice.² First, while nurses reported that monitors were necessary for most of their patients, participating nurses deemed few alarms clinically actionable and often chose not to further assess when they noted alarms outside of the room. This is in line with findings from prior studies suggesting that clinicians overvalue the contribution of monitoring systems to patient safety.^16,17 Second, while this finding occurred in a minority of the observations, the presence of family members at the patient’s bedside was cited by nurses as a rationale for whether they responded to alarms. While family members are capable of identifying safety issues,¹⁸ formal systems to engage them in patient safety and physiologic monitoring are lacking. Finally, clinical interventions or responses to the alerts of deteriorating patients, which best represented the original intent of CPMs, were rare and accounted for just 7% of the responses. Further work elucidating why physicians and nurses choose to use CPMs may be helpful to identify interventions to reduce inappropriate monitor use and highlight gaps in frontline staff knowledge about the benefits and risks of CPM use.

Our findings provide a novel understanding of previously observed phenomena, such as long response times or nonresponses in settings with high alarm rates.^4,10 Similar to that in a prior study conducted in the pediatric setting,¹¹ alarms with an observed response constituted a minority of the total alarms that occurred in our study. This finding has previously been attributed to mental fatigue, caregiver apathy, and desensitization.⁸ However, even though a minority of observed responses in our study included an intervention, the nurse had a rationale for why the alarm did or did not need a response. This behavior and the verbalized rationale indicate that in his/her opinion, not responding to the alarm was clinically appropriate. Study participants also reflected on the difficulties of responding to alarms given the monitor system setup, in which they may not always be capable of hearing alarms for their patients. Without data from nurses regarding the alarms that had no observed response, we can only speculate; however, based on our findings, each of these factors could contribute to nonresponse. Finally, while high numbers of false alarms have been posited as an underlying cause of alarm fatigue, we noted that a majority of nonresponse was reported to be related to other clinical factors. This relationship suggests that from the nurse’s perspective, a more applicable framework for understanding alarms would be based on clinical actionability⁴ over physiologic accuracy.

In total, our findings suggest that a multifaceted approach will be necessary to improve alarm response rates. These interventions should include adjusting parameters such that alarms are highly likely to indicate a need for intervention coupled with educational interventions addressing clinician knowledge of the alarm system and bias about the actionability of alarms may improve response rates. Changes in the monitoring system setup such that nurses can easily be notified when alarms occur may also be indicated, in addition to formally engaging patients and families around response to alarms. Although secondary notification systems (eg, alarms transmitted to individual clinician’s devices) are one solution, the utilization of these systems needs to be balanced with the risks of contributing to existing alarm fatigue and the need to appropriately tailor monitoring thresholds and strategies to patients.

Our study has several limitations. First, nurses may have responded in a way they perceive to be socially desirable, and studies using in-person observers are also prone to a Hawthorne-like effect,^19-21 where the nurse may have tried to respond more frequently to alarms than usual during observations. However, given that the majority of bedside alarms did not receive a response and a substantial number of responses involved no action, these effects were likely weak. Second, we were unable to assess which alarms were accurately reflecting the patient’s physiologic status and which were not; we were also unable to link observed alarm response to monitor-recorded alarms. Third, despite the use of silent observers and an actual, rather than a simulated, clinical setting, by virtue of the data collection method we likely captured a more deliberate thought process (so-called System 2 thinking)²² rather than the subconscious processes that may predominate when nurses respond to alarms in the course of clinical care (System 1 thinking).²² Despite this limitation, our study findings, which reflect a nurse’s in-the-moment thinking, remain relevant to guiding the improvement of monitoring systems, and the development of nurse-facing interventions and education. Finally, we studied a small, purposive sample of nurses at a single hospital. Our study sample impacts the generalizability of our results and precluded a detailed analysis of the effect of nurse- and patient-level variables.

We found that nurses often deemed that no response was necessary for CPM alarms. Nurses cited contextual factors, including the duration of alarms and the presence of other providers or parents in their decision-making. Few (7%) of the alarm responses in our study included a clinical intervention. The number of observed alarm responses constituted roughly a third of the alarms recorded by bedside CPMs during the study. This result supports concerns about the nurse’s capacity to hear and process all CPM alarms given system limitations and a heavy clinical workload. Subsequent steps should include staff education, reducing overall alarm rates with appropriate monitor use and actionable alarm thresholds, and ensuring that patient alarms are easily recognizable for frontline staff.

Disclosures

The authors have no conflicts of interest to disclose.

Funding

This work was supported by the Place Outcomes Research Award from the Cincinnati Children’s Research Foundation. Dr. Brady is supported by the Agency for Healthcare Research and Quality under Award Number K08HS23827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

Alarms from bedside continuous physiologic monitors (CPMs) occur frequently in children’s hospitals and can lead to harm. Recent studies conducted in children’s hospitals have identified alarm rates of up to 152 alarms per patient per day outside of the intensive care unit,^1-3 with as few as 1% of alarms being considered clinically important.⁴ Excessive alarms have been linked to alarm fatigue, when providers become desensitized to and may miss alarms indicating impending patient deterioration. Alarm fatigue has been identified by national patient safety organizations as a patient safety concern given the risk of patient harm.^5-7 Despite these concerns, CPMs are routinely used: up to 48% of pediatric patients in nonintensive care units at children’s hospitals are monitored.²

Although the low number of alarms that receive responses has been well-described,^8,9 the reasons why clinicians do or do not respond to alarms are unclear. A study conducted in an adult perioperative unit noted prolonged nurse response times for patients with high alarm rates.¹⁰ A second study conducted in the pediatric inpatient setting demonstrated a dose-response effect and noted progressively prolonged nurse response times with increased rates of nonactionable alarms.^4,11 Findings from another study suggested that underlying factors are highly complex and may be a result of excessive alarms, clinician characteristics, and working conditions (eg, workload and unit noise level).¹² Evidence also suggests that humans have difficulty distinguishing the importance of alarms in situations where multiple alarm tones are used, a common scenario in hospitals.^13,14 Understanding the factors that contribute to clinicians responding or not responding to CPM alarms will be crucial for addressing this serious patient safety issue.

An enhanced understanding of why nurses respond to alarms in daily practice will inform intervention development and improvement work. In the long term, this information could help improve systems for monitoring pediatric inpatients that are less prone to issues with alarm fatigue. The objective of this qualitative study, which employed structured observation, was to describe how bedside nurses think about and act upon bedside monitor alarms in a general pediatric inpatient unit.

Study Design and Setting

This prospective observational study took place on a 48-bed hospital medicine unit at a large, freestanding children’s hospital with >650 beds and >19,000 annual admissions. General Electric (Little Chalfont, United Kingdom) physiologic monitors (models Dash 3000, 4000, and 5000) were used at the time of the study, and nurses could be notified of monitor alarms in four ways: First, an in-room auditory alarm sounds. Second, a light positioned above the door outside of each patient room blinks for alarms that are at a “warning” or “critical level” (eg ventricular tachycardia or low oxygen saturation). Third, audible alarms occur at the unit’s central monitoring station. Lastly, another staff member can notify the patient’s nurse via in-person conversion or secure smart phone communication. On the study unit, CPMs are initiated and discontinued through a physician order.

This study was reviewed and approved by the hospital’s institutional review board.

Study Population

We used a purposive recruitment strategy to enroll bedside nurses working on general hospital medicine units, stratified to ensure varying levels of experience and primary shifts (eg, day vs night). We planned to conduct approximately two observations with each participating nurse and to continue collecting data until we could no longer identify new insights in terms of responses to alarms (ie, thematic saturation¹⁵). Observations were targeted to cover times of day that coincided with increased rates of distraction. These times included just prior to and after the morning and evening change of shifts (7:00 am and 7:00 pm), during morning rounds (8:00 am-12:00 pm), and heavy admission times (12:00 pm-10:00 pm). After written informed consent, a nurse was eligible for observation during his/her shift if he/she was caring for at least one monitored patient. Enrolled nurses were made aware of the general study topic but were blinded to the study team’s hypotheses.

Data Sources

Prior to data collection, the research team, which consisted of physicians, bedside nurses, research coordinators, and a human factors expert, created a system for categorizing alarm responses. Categories for observed responses were based on the location and corresponding action taken. Initial categories were developed a priori from existing literature and expanded through input from the multidisciplinary study team, then vetted with bedside staff, and finally pilot tested through >4 hours of observations, thus producing the final categories. These categories were entered into a work-sampling program (WorkStudy by Quetech Ltd., Waterloo, Ontario, Canada) to facilitate quick data recording during observations.

The hospital uses a central alarm collection software (BedMasterEx by Anandic Medical Systems, Feuerthalen, Switzerland), which permitted the collection of date, time, trigger (eg, high heart rate), and level (eg, crisis, warning) of the generated CPM alarms. Alarms collected are based on thresholds preset at the bedside monitor. The central collection software does not differentiate between accurate (eg, correctly representing the physiologic state of the patient) and inaccurate alarms.

Observation Procedure

At the time of observation, nurse demographic information (eg, primary shift worked and years working as a nurse) was obtained. A brief preobservation questionnaire was administered to collect patient information (eg, age and diagnosis) and the nurses’ perspectives on the necessity of monitors for each monitored patient in his/her care.

The observer shadowed the nurse for a two-hour block of his/her shift. During this time, nurses were instructed to “think aloud” as they responded to alarms (eg, “I notice the oxygen saturation monitor alarming off, but the probe has fallen off”). A trained observer (AML or KMT) recorded responses verbalized by the nurse and his/her reaction by selecting the appropriate category using the work-sampling software. Data were also collected on the vital sign associated with the alarm (eg, heart rate). Moreover, the observer kept written notes to provide context for electronically recorded data. Alarms that were not verbalized by the nurse were not counted. Similarly, alarms that were noted outside of the room by the nurse were not classified by vital sign unless the nurse confirmed with the bedside monitor. Observers did not adjudicate the accuracy of the alarms. The session was stopped if monitors were discontinued during the observation period. Alarm data generated by the bedside monitor were pulled for each patient room after observations were completed.

Analysis

Descriptive statistics were used to assess the percentage of each nurse response category and each alarm type (eg, heart rate and respiratory rate). The observed alarm rate was calculated by taking the total number of observed alarms (ie, alarms noted by the nurse) divided by the total number of patient-hours observed. The monitor-generated alarm rate was calculated by taking the total number of alarms from the bedside-alarm generated data divided by the number of patient-hours observed.

Electronically recorded observations using the work-sampling program were cross-referenced with hand-written field notes to assess for any discrepancies or identify relevant events not captured by the program. Three study team members (AML, KMT, and ACS) reviewed each observation independently and compared field notes to ensure accurate categorization. Discrepancies were referred to the larger study group in cases of uncertainty.

Nine nurses had monitored patients during the available observations and participated in 19 observation sessions, which included 35 monitored patients for a total of 61.3 patient-hours of observation. Nurses were observed for a median of two times each (range 1-4). The median number of monitored patients during a single observation session was two (range 1-3). Observed nurses were female with a median of eight years of experience (range 0.5-26 years). Patients represented a broad range of age categories and were hospitalized with a variety of diagnoses (Table). Nurses, when queried at the start of the observation, felt that monitors were necessary for 29 (82.9%) of the observed patients given either patient condition or unit policy.

A total of 207 observed nurse responses to alarms occurred during the study period for a rate of 3.4 responses per patient per hour. Of the total number of responses, 45 (21.7%) were noted outside of a patient room, and in 15 (33.3%) the nurse chose to go to the room. The other 162 were recorded when the nurse was present in the room when the alarm activated. Of the 177 in-person nurse responses, 50 were related to a pulse oximetry alarm, 66 were related to a heart rate alarm, and 61 were related to a respiratory rate alarm. The most common observed in-person response to an alarm involved the nurse judging that no intervention was necessary (n = 152, 73.1%). Only 14 (7% of total responses) observed in-person responses involved a clinical intervention, such as suctioning or titrating supplemental oxygen. Findings are summarized in the Figure and describe nurse-verbalized reasons to further assess (or not) and then whether the nurse chose to take action (or not) after an alarm.

We characterized responses to physiologic monitor alarms by a group of nurses with a range of experience levels. We found that most nurse responses to alarms in continuously monitored general pediatric patients involved no intervention, and further assessment was often not conducted for alarms that occurred outside of the room if the nurse noted otherwise reassuring clinical context. Observed responses occurred for 36% of alarms during the study period when compared with bedside monitor-alarm generated data. Overall, only 14 clinical interventions were noted among the observed responses. Nurses noted that they felt the monitors were necessary for 82.9% of monitored patients because of the clinical context or because of unit policy.

Our study findings highlight some potential contradictions in the current widespread use of CPMs in general pediatric units and how clinicians respond to them in practice.² First, while nurses reported that monitors were necessary for most of their patients, participating nurses deemed few alarms clinically actionable and often chose not to further assess when they noted alarms outside of the room. This is in line with findings from prior studies suggesting that clinicians overvalue the contribution of monitoring systems to patient safety.^16,17 Second, while this finding occurred in a minority of the observations, the presence of family members at the patient’s bedside was cited by nurses as a rationale for whether they responded to alarms. While family members are capable of identifying safety issues,¹⁸ formal systems to engage them in patient safety and physiologic monitoring are lacking. Finally, clinical interventions or responses to the alerts of deteriorating patients, which best represented the original intent of CPMs, were rare and accounted for just 7% of the responses. Further work elucidating why physicians and nurses choose to use CPMs may be helpful to identify interventions to reduce inappropriate monitor use and highlight gaps in frontline staff knowledge about the benefits and risks of CPM use.

Our findings provide a novel understanding of previously observed phenomena, such as long response times or nonresponses in settings with high alarm rates.^4,10 Similar to that in a prior study conducted in the pediatric setting,¹¹ alarms with an observed response constituted a minority of the total alarms that occurred in our study. This finding has previously been attributed to mental fatigue, caregiver apathy, and desensitization.⁸ However, even though a minority of observed responses in our study included an intervention, the nurse had a rationale for why the alarm did or did not need a response. This behavior and the verbalized rationale indicate that in his/her opinion, not responding to the alarm was clinically appropriate. Study participants also reflected on the difficulties of responding to alarms given the monitor system setup, in which they may not always be capable of hearing alarms for their patients. Without data from nurses regarding the alarms that had no observed response, we can only speculate; however, based on our findings, each of these factors could contribute to nonresponse. Finally, while high numbers of false alarms have been posited as an underlying cause of alarm fatigue, we noted that a majority of nonresponse was reported to be related to other clinical factors. This relationship suggests that from the nurse’s perspective, a more applicable framework for understanding alarms would be based on clinical actionability⁴ over physiologic accuracy.

In total, our findings suggest that a multifaceted approach will be necessary to improve alarm response rates. These interventions should include adjusting parameters such that alarms are highly likely to indicate a need for intervention coupled with educational interventions addressing clinician knowledge of the alarm system and bias about the actionability of alarms may improve response rates. Changes in the monitoring system setup such that nurses can easily be notified when alarms occur may also be indicated, in addition to formally engaging patients and families around response to alarms. Although secondary notification systems (eg, alarms transmitted to individual clinician’s devices) are one solution, the utilization of these systems needs to be balanced with the risks of contributing to existing alarm fatigue and the need to appropriately tailor monitoring thresholds and strategies to patients.

Our study has several limitations. First, nurses may have responded in a way they perceive to be socially desirable, and studies using in-person observers are also prone to a Hawthorne-like effect,^19-21 where the nurse may have tried to respond more frequently to alarms than usual during observations. However, given that the majority of bedside alarms did not receive a response and a substantial number of responses involved no action, these effects were likely weak. Second, we were unable to assess which alarms were accurately reflecting the patient’s physiologic status and which were not; we were also unable to link observed alarm response to monitor-recorded alarms. Third, despite the use of silent observers and an actual, rather than a simulated, clinical setting, by virtue of the data collection method we likely captured a more deliberate thought process (so-called System 2 thinking)²² rather than the subconscious processes that may predominate when nurses respond to alarms in the course of clinical care (System 1 thinking).²² Despite this limitation, our study findings, which reflect a nurse’s in-the-moment thinking, remain relevant to guiding the improvement of monitoring systems, and the development of nurse-facing interventions and education. Finally, we studied a small, purposive sample of nurses at a single hospital. Our study sample impacts the generalizability of our results and precluded a detailed analysis of the effect of nurse- and patient-level variables.

We found that nurses often deemed that no response was necessary for CPM alarms. Nurses cited contextual factors, including the duration of alarms and the presence of other providers or parents in their decision-making. Few (7%) of the alarm responses in our study included a clinical intervention. The number of observed alarm responses constituted roughly a third of the alarms recorded by bedside CPMs during the study. This result supports concerns about the nurse’s capacity to hear and process all CPM alarms given system limitations and a heavy clinical workload. Subsequent steps should include staff education, reducing overall alarm rates with appropriate monitor use and actionable alarm thresholds, and ensuring that patient alarms are easily recognizable for frontline staff.

Disclosures

The authors have no conflicts of interest to disclose.

Funding

This work was supported by the Place Outcomes Research Award from the Cincinnati Children’s Research Foundation. Dr. Brady is supported by the Agency for Healthcare Research and Quality under Award Number K08HS23827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

Alarms from bedside continuous physiologic monitors (CPMs) occur frequently in children’s hospitals and can lead to harm. Recent studies conducted in children’s hospitals have identified alarm rates of up to 152 alarms per patient per day outside of the intensive care unit,^1-3 with as few as 1% of alarms being considered clinically important.⁴ Excessive alarms have been linked to alarm fatigue, when providers become desensitized to and may miss alarms indicating impending patient deterioration. Alarm fatigue has been identified by national patient safety organizations as a patient safety concern given the risk of patient harm.^5-7 Despite these concerns, CPMs are routinely used: up to 48% of pediatric patients in nonintensive care units at children’s hospitals are monitored.²

Although the low number of alarms that receive responses has been well-described,^8,9 the reasons why clinicians do or do not respond to alarms are unclear. A study conducted in an adult perioperative unit noted prolonged nurse response times for patients with high alarm rates.¹⁰ A second study conducted in the pediatric inpatient setting demonstrated a dose-response effect and noted progressively prolonged nurse response times with increased rates of nonactionable alarms.^4,11 Findings from another study suggested that underlying factors are highly complex and may be a result of excessive alarms, clinician characteristics, and working conditions (eg, workload and unit noise level).¹² Evidence also suggests that humans have difficulty distinguishing the importance of alarms in situations where multiple alarm tones are used, a common scenario in hospitals.^13,14 Understanding the factors that contribute to clinicians responding or not responding to CPM alarms will be crucial for addressing this serious patient safety issue.

An enhanced understanding of why nurses respond to alarms in daily practice will inform intervention development and improvement work. In the long term, this information could help improve systems for monitoring pediatric inpatients that are less prone to issues with alarm fatigue. The objective of this qualitative study, which employed structured observation, was to describe how bedside nurses think about and act upon bedside monitor alarms in a general pediatric inpatient unit.

Study Design and Setting

This prospective observational study took place on a 48-bed hospital medicine unit at a large, freestanding children’s hospital with >650 beds and >19,000 annual admissions. General Electric (Little Chalfont, United Kingdom) physiologic monitors (models Dash 3000, 4000, and 5000) were used at the time of the study, and nurses could be notified of monitor alarms in four ways: First, an in-room auditory alarm sounds. Second, a light positioned above the door outside of each patient room blinks for alarms that are at a “warning” or “critical level” (eg ventricular tachycardia or low oxygen saturation). Third, audible alarms occur at the unit’s central monitoring station. Lastly, another staff member can notify the patient’s nurse via in-person conversion or secure smart phone communication. On the study unit, CPMs are initiated and discontinued through a physician order.

This study was reviewed and approved by the hospital’s institutional review board.

Study Population

We used a purposive recruitment strategy to enroll bedside nurses working on general hospital medicine units, stratified to ensure varying levels of experience and primary shifts (eg, day vs night). We planned to conduct approximately two observations with each participating nurse and to continue collecting data until we could no longer identify new insights in terms of responses to alarms (ie, thematic saturation¹⁵). Observations were targeted to cover times of day that coincided with increased rates of distraction. These times included just prior to and after the morning and evening change of shifts (7:00 am and 7:00 pm), during morning rounds (8:00 am-12:00 pm), and heavy admission times (12:00 pm-10:00 pm). After written informed consent, a nurse was eligible for observation during his/her shift if he/she was caring for at least one monitored patient. Enrolled nurses were made aware of the general study topic but were blinded to the study team’s hypotheses.

Data Sources

Prior to data collection, the research team, which consisted of physicians, bedside nurses, research coordinators, and a human factors expert, created a system for categorizing alarm responses. Categories for observed responses were based on the location and corresponding action taken. Initial categories were developed a priori from existing literature and expanded through input from the multidisciplinary study team, then vetted with bedside staff, and finally pilot tested through >4 hours of observations, thus producing the final categories. These categories were entered into a work-sampling program (WorkStudy by Quetech Ltd., Waterloo, Ontario, Canada) to facilitate quick data recording during observations.

The hospital uses a central alarm collection software (BedMasterEx by Anandic Medical Systems, Feuerthalen, Switzerland), which permitted the collection of date, time, trigger (eg, high heart rate), and level (eg, crisis, warning) of the generated CPM alarms. Alarms collected are based on thresholds preset at the bedside monitor. The central collection software does not differentiate between accurate (eg, correctly representing the physiologic state of the patient) and inaccurate alarms.

Observation Procedure

At the time of observation, nurse demographic information (eg, primary shift worked and years working as a nurse) was obtained. A brief preobservation questionnaire was administered to collect patient information (eg, age and diagnosis) and the nurses’ perspectives on the necessity of monitors for each monitored patient in his/her care.

The observer shadowed the nurse for a two-hour block of his/her shift. During this time, nurses were instructed to “think aloud” as they responded to alarms (eg, “I notice the oxygen saturation monitor alarming off, but the probe has fallen off”). A trained observer (AML or KMT) recorded responses verbalized by the nurse and his/her reaction by selecting the appropriate category using the work-sampling software. Data were also collected on the vital sign associated with the alarm (eg, heart rate). Moreover, the observer kept written notes to provide context for electronically recorded data. Alarms that were not verbalized by the nurse were not counted. Similarly, alarms that were noted outside of the room by the nurse were not classified by vital sign unless the nurse confirmed with the bedside monitor. Observers did not adjudicate the accuracy of the alarms. The session was stopped if monitors were discontinued during the observation period. Alarm data generated by the bedside monitor were pulled for each patient room after observations were completed.

Analysis

Descriptive statistics were used to assess the percentage of each nurse response category and each alarm type (eg, heart rate and respiratory rate). The observed alarm rate was calculated by taking the total number of observed alarms (ie, alarms noted by the nurse) divided by the total number of patient-hours observed. The monitor-generated alarm rate was calculated by taking the total number of alarms from the bedside-alarm generated data divided by the number of patient-hours observed.

Electronically recorded observations using the work-sampling program were cross-referenced with hand-written field notes to assess for any discrepancies or identify relevant events not captured by the program. Three study team members (AML, KMT, and ACS) reviewed each observation independently and compared field notes to ensure accurate categorization. Discrepancies were referred to the larger study group in cases of uncertainty.

Nine nurses had monitored patients during the available observations and participated in 19 observation sessions, which included 35 monitored patients for a total of 61.3 patient-hours of observation. Nurses were observed for a median of two times each (range 1-4). The median number of monitored patients during a single observation session was two (range 1-3). Observed nurses were female with a median of eight years of experience (range 0.5-26 years). Patients represented a broad range of age categories and were hospitalized with a variety of diagnoses (Table). Nurses, when queried at the start of the observation, felt that monitors were necessary for 29 (82.9%) of the observed patients given either patient condition or unit policy.

A total of 207 observed nurse responses to alarms occurred during the study period for a rate of 3.4 responses per patient per hour. Of the total number of responses, 45 (21.7%) were noted outside of a patient room, and in 15 (33.3%) the nurse chose to go to the room. The other 162 were recorded when the nurse was present in the room when the alarm activated. Of the 177 in-person nurse responses, 50 were related to a pulse oximetry alarm, 66 were related to a heart rate alarm, and 61 were related to a respiratory rate alarm. The most common observed in-person response to an alarm involved the nurse judging that no intervention was necessary (n = 152, 73.1%). Only 14 (7% of total responses) observed in-person responses involved a clinical intervention, such as suctioning or titrating supplemental oxygen. Findings are summarized in the Figure and describe nurse-verbalized reasons to further assess (or not) and then whether the nurse chose to take action (or not) after an alarm.

We characterized responses to physiologic monitor alarms by a group of nurses with a range of experience levels. We found that most nurse responses to alarms in continuously monitored general pediatric patients involved no intervention, and further assessment was often not conducted for alarms that occurred outside of the room if the nurse noted otherwise reassuring clinical context. Observed responses occurred for 36% of alarms during the study period when compared with bedside monitor-alarm generated data. Overall, only 14 clinical interventions were noted among the observed responses. Nurses noted that they felt the monitors were necessary for 82.9% of monitored patients because of the clinical context or because of unit policy.

Our study findings highlight some potential contradictions in the current widespread use of CPMs in general pediatric units and how clinicians respond to them in practice.² First, while nurses reported that monitors were necessary for most of their patients, participating nurses deemed few alarms clinically actionable and often chose not to further assess when they noted alarms outside of the room. This is in line with findings from prior studies suggesting that clinicians overvalue the contribution of monitoring systems to patient safety.^16,17 Second, while this finding occurred in a minority of the observations, the presence of family members at the patient’s bedside was cited by nurses as a rationale for whether they responded to alarms. While family members are capable of identifying safety issues,¹⁸ formal systems to engage them in patient safety and physiologic monitoring are lacking. Finally, clinical interventions or responses to the alerts of deteriorating patients, which best represented the original intent of CPMs, were rare and accounted for just 7% of the responses. Further work elucidating why physicians and nurses choose to use CPMs may be helpful to identify interventions to reduce inappropriate monitor use and highlight gaps in frontline staff knowledge about the benefits and risks of CPM use.

Our findings provide a novel understanding of previously observed phenomena, such as long response times or nonresponses in settings with high alarm rates.^4,10 Similar to that in a prior study conducted in the pediatric setting,¹¹ alarms with an observed response constituted a minority of the total alarms that occurred in our study. This finding has previously been attributed to mental fatigue, caregiver apathy, and desensitization.⁸ However, even though a minority of observed responses in our study included an intervention, the nurse had a rationale for why the alarm did or did not need a response. This behavior and the verbalized rationale indicate that in his/her opinion, not responding to the alarm was clinically appropriate. Study participants also reflected on the difficulties of responding to alarms given the monitor system setup, in which they may not always be capable of hearing alarms for their patients. Without data from nurses regarding the alarms that had no observed response, we can only speculate; however, based on our findings, each of these factors could contribute to nonresponse. Finally, while high numbers of false alarms have been posited as an underlying cause of alarm fatigue, we noted that a majority of nonresponse was reported to be related to other clinical factors. This relationship suggests that from the nurse’s perspective, a more applicable framework for understanding alarms would be based on clinical actionability⁴ over physiologic accuracy.

In total, our findings suggest that a multifaceted approach will be necessary to improve alarm response rates. These interventions should include adjusting parameters such that alarms are highly likely to indicate a need for intervention coupled with educational interventions addressing clinician knowledge of the alarm system and bias about the actionability of alarms may improve response rates. Changes in the monitoring system setup such that nurses can easily be notified when alarms occur may also be indicated, in addition to formally engaging patients and families around response to alarms. Although secondary notification systems (eg, alarms transmitted to individual clinician’s devices) are one solution, the utilization of these systems needs to be balanced with the risks of contributing to existing alarm fatigue and the need to appropriately tailor monitoring thresholds and strategies to patients.

Our study has several limitations. First, nurses may have responded in a way they perceive to be socially desirable, and studies using in-person observers are also prone to a Hawthorne-like effect,^19-21 where the nurse may have tried to respond more frequently to alarms than usual during observations. However, given that the majority of bedside alarms did not receive a response and a substantial number of responses involved no action, these effects were likely weak. Second, we were unable to assess which alarms were accurately reflecting the patient’s physiologic status and which were not; we were also unable to link observed alarm response to monitor-recorded alarms. Third, despite the use of silent observers and an actual, rather than a simulated, clinical setting, by virtue of the data collection method we likely captured a more deliberate thought process (so-called System 2 thinking)²² rather than the subconscious processes that may predominate when nurses respond to alarms in the course of clinical care (System 1 thinking).²² Despite this limitation, our study findings, which reflect a nurse’s in-the-moment thinking, remain relevant to guiding the improvement of monitoring systems, and the development of nurse-facing interventions and education. Finally, we studied a small, purposive sample of nurses at a single hospital. Our study sample impacts the generalizability of our results and precluded a detailed analysis of the effect of nurse- and patient-level variables.

We found that nurses often deemed that no response was necessary for CPM alarms. Nurses cited contextual factors, including the duration of alarms and the presence of other providers or parents in their decision-making. Few (7%) of the alarm responses in our study included a clinical intervention. The number of observed alarm responses constituted roughly a third of the alarms recorded by bedside CPMs during the study. This result supports concerns about the nurse’s capacity to hear and process all CPM alarms given system limitations and a heavy clinical workload. Subsequent steps should include staff education, reducing overall alarm rates with appropriate monitor use and actionable alarm thresholds, and ensuring that patient alarms are easily recognizable for frontline staff.

Disclosures

The authors have no conflicts of interest to disclose.

Funding

This work was supported by the Place Outcomes Research Award from the Cincinnati Children’s Research Foundation. Dr. Brady is supported by the Agency for Healthcare Research and Quality under Award Number K08HS23827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

Automated early warning systems (EWSs) use data inputs to recognize clinical states requiring time-sensitive intervention and then generate notifications through different modalities to clinicians. EWSs serve as common tools for improving the recognition and treatment of important clinical states such as sepsis. However, despite the early enthusiasm, these warning systems have often yielded disappointing outcomes. In sepsis, for example, EWSs have shown mixed results in clinical trials, and concerns regarding the overuse of EWSs in diagnosing sepsis have grown.^1-4 We argue that inattention to the importance of timing in EWS training and evaluation provides one reason that EWSs have underperformed. Thus, to improve care, a warning system must not only identify the clinical state accurately, but it must also do so in a sufficiently timely manner to implement the associated interventions, such as administration of antibiotics for sepsis. Although the literature has occasionally highlighted the importance of timing in electronic surveillance systems, no one has linked the temporal dependence of performance metrics and intervention feasibility to the failure of such warning systems and explained how to operationalize timing in their development.^5-8 Using sepsis as an example, we explain why timing is important and propose new metrics and strategies for training and evaluating EWS models. EWSs are divided into two types: detection systems that recognize critical illnesses at a particular moment and prediction systems that estimate risk of deterioration over varying time frames.⁹ We focus primarily on detection systems, but our analysis is also important for prediction systems, which we will discuss in the last section.

EWS metrics have evolved from focusing on crude measures of discrimination to more clinically relevant metrics, such as the positive predictive value (PPV). The common performance metrics, including the c-statistic, evaluate the performance of EWSs in distinguishing events from nonevents, such as the presence or absence of sepsis in hospitalized patients. However, the c-statistic does not account for disease prevalence. A given c-statistic is compatible with a wide range of PPVs; a low PPV may limit an EWS’s usefulness to promote interventions and generate increased alert fatigue.¹⁰

However, the PPV, although important, provides no information on the timing of state recognition in relation to clinical time zero. Time zero is the first moment at which a critical state can be recognized based on available data and current medical science. Different approaches, including laboratory values, clinical assessments, retrospective chart reviews, triage times, and others, have been used to measure time zero.^8,11-13 All these approaches feature advantages and disadvantages; the evaluation of timing will exhibit sensitivity to the approach used.¹⁴ Further work is needed to gain additional insights into the measurement of time zero.

a) 50% true cases of sepsis, with a mean time of 35 minutes after clinical time zero;

b) 50% true cases, with a mean time of 60 minutes before clinical time zero (prediction EWS);

c) 50% true cases of sepsis, with a mean time of 1.3 days since clinical time zero, but with 70% of these cases undiagnosed at the time of EWS detection;

d) 50% true cases of cases, with mean time of 1.3 days since clinical time zero, that is, all cases among those promptly detected and treated through routine clinician oversight.

Each of these situations features differing clinical utility to help meet the hospital objective of increasing early administration of antibiotics. More generally, three dimensions of timing are important for detection systems. The first dimension is the timing of detection relative to time zero. The second is the timing relative to ”real-world” clinician detection. The third is timing with respect to the associated clinical objective. For a given PPV, an EWS performs better when detecting a state (1) at, near, or in advance of time zero, (2) prior to clinician detection, and (3) sufficiently in advance of an operational objective to promote change. On the other hand, when an EWS consistently sends alerts after clinician action, it serves a lesser purpose and risks causing alert fatigue; such cases have been described in studies.¹⁵

Acknowledging the importance of timing features implications for researchers and health system leaders. Researchers who develop EWS should include how these systems perform relative to both time zero and critical milestones in the clinical course. Operational leadership should understand the trade-offs that occur between alert fatigue (through lower PPV at the margin with earlier detection) and lead time to implement an intervention. Navigating these trade-offs involves a complex organizational decision. The “number needed to evaluate” is one way to quantify this fatigue factor.¹⁶ Such a measure gives a sense of the number of cases a clinician will need to evaluate per event. Collaborations between clinical leadership, operational leadership, and data scientists are needed to determine how to evaluate individual systems.

A good metric should capture the three important dimensions of timing while retaining intuitiveness to clinicians and leadership. One graphical option involves plotting the PPVs over time and relative to the clinical state evolution (Figure). This PPV-over-time curve shows when true positives occur relative to the time course of sepsis, including the three major dimensions of timing. This curve can also show a “clinically important window (CIW)”, which is bounded on the right by the latest point in time when recognition could still meet the clinical objective. For sepsis, the curve might be bounded at 2.5 hours to meet an objective of antibiotics within three hours, with the assumption that 0.5 hour is needed for a response. For detection systems, the window would be bounded on the left by clinical time zero. The graph can also designate the point when most cases of sepsis have been recognized clinically with historical data. The Figure depicts an example curve for a detection model.

The metrics derived from this curve may be used alongside the PPV for training and evaluation. Often, adjusting the PPV for its relationship to time zero and the CIW will aid in recognizing the existence of a time beyond which detection fails to help achieve the intended intervention. Detection beyond the window should not credited as a true positive if it fails to facilitate the objective. One option is to credit detection at or before time zero as one and discount later detection by the delay from time zero. More specifically, a true positive could be discounted by the difference between the end of the CIW and the moment of detection divided by the CIW length. This discounted PPV could be displayed alongside the PPV to gauge the temporal dimension of performance and be used for training.

The use of timing places additional demands on validation owing to the need for a time-based gold standard. In such a case, the unit of analysis in system development might not be the patient encounter but rather the patient-hour or patient-15-minute epoch, depending on how frequently the EWS updates risk information and may alert. By contrast, the sepsis detection models used in administrative databases rely on an encounter-level PPV, which provides more limited information compared with real-time EWSs.¹⁷ When time zero cannot be measured, alternatives may be used to capture several dimensions of timing; these alternatives include measurement of the percentage of cases that recognize the event prior to clinicians.¹⁵

Detection systems face the limitation that they lack the capability to identify a state before its occurrence. Prediction systems are more likely to be actionable, as they provide more lead time for intervention, but accurate prediction models are also more difficult to develop. With a predictive system, an additional dimension of timing becomes important: the time horizon for prediction. Prediction models may be trained to recognize a state within a specific time frame (eg, 6, 12, or 24 hours), and test characteristics, including PPV, may vary with the window.¹⁸ A given PPV (of eventual development of sepsis) is compatible with varying time windows and thus again lacks important information on performance.

The timing relative to clinical time zero remains important for prediction. For a predictive EWS, the graph in the figure may be expected to shift to the left. Models with good performance will occasionally send an alert after time zero. For a prediction system with a time horizon of six hours, it is more useful to have alerts occur a mean time of four hours prior to time zero than four minutes prior.

Improving the clinical utility of EWSs requires better measurement of timing. Researchers should incorporate timing into system development, and operational leaders should be cognizant of timing during implementation. Specific steps should include devising better strategies to estimate the relationship of state recognition to clinical time zero and developing methods to discount recognition when it occurs too late to be actionable.

Disclosures

Dr. Rolnick is a consultant to Tuple Health, Inc. and was previously a part-time employee of Acumen, LLC. Dr. Weissman has nothing to disclose.

Automated early warning systems (EWSs) use data inputs to recognize clinical states requiring time-sensitive intervention and then generate notifications through different modalities to clinicians. EWSs serve as common tools for improving the recognition and treatment of important clinical states such as sepsis. However, despite the early enthusiasm, these warning systems have often yielded disappointing outcomes. In sepsis, for example, EWSs have shown mixed results in clinical trials, and concerns regarding the overuse of EWSs in diagnosing sepsis have grown.^1-4 We argue that inattention to the importance of timing in EWS training and evaluation provides one reason that EWSs have underperformed. Thus, to improve care, a warning system must not only identify the clinical state accurately, but it must also do so in a sufficiently timely manner to implement the associated interventions, such as administration of antibiotics for sepsis. Although the literature has occasionally highlighted the importance of timing in electronic surveillance systems, no one has linked the temporal dependence of performance metrics and intervention feasibility to the failure of such warning systems and explained how to operationalize timing in their development.^5-8 Using sepsis as an example, we explain why timing is important and propose new metrics and strategies for training and evaluating EWS models. EWSs are divided into two types: detection systems that recognize critical illnesses at a particular moment and prediction systems that estimate risk of deterioration over varying time frames.⁹ We focus primarily on detection systems, but our analysis is also important for prediction systems, which we will discuss in the last section.

EWS metrics have evolved from focusing on crude measures of discrimination to more clinically relevant metrics, such as the positive predictive value (PPV). The common performance metrics, including the c-statistic, evaluate the performance of EWSs in distinguishing events from nonevents, such as the presence or absence of sepsis in hospitalized patients. However, the c-statistic does not account for disease prevalence. A given c-statistic is compatible with a wide range of PPVs; a low PPV may limit an EWS’s usefulness to promote interventions and generate increased alert fatigue.¹⁰

However, the PPV, although important, provides no information on the timing of state recognition in relation to clinical time zero. Time zero is the first moment at which a critical state can be recognized based on available data and current medical science. Different approaches, including laboratory values, clinical assessments, retrospective chart reviews, triage times, and others, have been used to measure time zero.^8,11-13 All these approaches feature advantages and disadvantages; the evaluation of timing will exhibit sensitivity to the approach used.¹⁴ Further work is needed to gain additional insights into the measurement of time zero.

a) 50% true cases of sepsis, with a mean time of 35 minutes after clinical time zero;

b) 50% true cases, with a mean time of 60 minutes before clinical time zero (prediction EWS);

c) 50% true cases of sepsis, with a mean time of 1.3 days since clinical time zero, but with 70% of these cases undiagnosed at the time of EWS detection;

d) 50% true cases of cases, with mean time of 1.3 days since clinical time zero, that is, all cases among those promptly detected and treated through routine clinician oversight.

Each of these situations features differing clinical utility to help meet the hospital objective of increasing early administration of antibiotics. More generally, three dimensions of timing are important for detection systems. The first dimension is the timing of detection relative to time zero. The second is the timing relative to ”real-world” clinician detection. The third is timing with respect to the associated clinical objective. For a given PPV, an EWS performs better when detecting a state (1) at, near, or in advance of time zero, (2) prior to clinician detection, and (3) sufficiently in advance of an operational objective to promote change. On the other hand, when an EWS consistently sends alerts after clinician action, it serves a lesser purpose and risks causing alert fatigue; such cases have been described in studies.¹⁵

Acknowledging the importance of timing features implications for researchers and health system leaders. Researchers who develop EWS should include how these systems perform relative to both time zero and critical milestones in the clinical course. Operational leadership should understand the trade-offs that occur between alert fatigue (through lower PPV at the margin with earlier detection) and lead time to implement an intervention. Navigating these trade-offs involves a complex organizational decision. The “number needed to evaluate” is one way to quantify this fatigue factor.¹⁶ Such a measure gives a sense of the number of cases a clinician will need to evaluate per event. Collaborations between clinical leadership, operational leadership, and data scientists are needed to determine how to evaluate individual systems.

A good metric should capture the three important dimensions of timing while retaining intuitiveness to clinicians and leadership. One graphical option involves plotting the PPVs over time and relative to the clinical state evolution (Figure). This PPV-over-time curve shows when true positives occur relative to the time course of sepsis, including the three major dimensions of timing. This curve can also show a “clinically important window (CIW)”, which is bounded on the right by the latest point in time when recognition could still meet the clinical objective. For sepsis, the curve might be bounded at 2.5 hours to meet an objective of antibiotics within three hours, with the assumption that 0.5 hour is needed for a response. For detection systems, the window would be bounded on the left by clinical time zero. The graph can also designate the point when most cases of sepsis have been recognized clinically with historical data. The Figure depicts an example curve for a detection model.

The metrics derived from this curve may be used alongside the PPV for training and evaluation. Often, adjusting the PPV for its relationship to time zero and the CIW will aid in recognizing the existence of a time beyond which detection fails to help achieve the intended intervention. Detection beyond the window should not credited as a true positive if it fails to facilitate the objective. One option is to credit detection at or before time zero as one and discount later detection by the delay from time zero. More specifically, a true positive could be discounted by the difference between the end of the CIW and the moment of detection divided by the CIW length. This discounted PPV could be displayed alongside the PPV to gauge the temporal dimension of performance and be used for training.

The use of timing places additional demands on validation owing to the need for a time-based gold standard. In such a case, the unit of analysis in system development might not be the patient encounter but rather the patient-hour or patient-15-minute epoch, depending on how frequently the EWS updates risk information and may alert. By contrast, the sepsis detection models used in administrative databases rely on an encounter-level PPV, which provides more limited information compared with real-time EWSs.¹⁷ When time zero cannot be measured, alternatives may be used to capture several dimensions of timing; these alternatives include measurement of the percentage of cases that recognize the event prior to clinicians.¹⁵

Detection systems face the limitation that they lack the capability to identify a state before its occurrence. Prediction systems are more likely to be actionable, as they provide more lead time for intervention, but accurate prediction models are also more difficult to develop. With a predictive system, an additional dimension of timing becomes important: the time horizon for prediction. Prediction models may be trained to recognize a state within a specific time frame (eg, 6, 12, or 24 hours), and test characteristics, including PPV, may vary with the window.¹⁸ A given PPV (of eventual development of sepsis) is compatible with varying time windows and thus again lacks important information on performance.

The timing relative to clinical time zero remains important for prediction. For a predictive EWS, the graph in the figure may be expected to shift to the left. Models with good performance will occasionally send an alert after time zero. For a prediction system with a time horizon of six hours, it is more useful to have alerts occur a mean time of four hours prior to time zero than four minutes prior.

Improving the clinical utility of EWSs requires better measurement of timing. Researchers should incorporate timing into system development, and operational leaders should be cognizant of timing during implementation. Specific steps should include devising better strategies to estimate the relationship of state recognition to clinical time zero and developing methods to discount recognition when it occurs too late to be actionable.

Disclosures

Dr. Rolnick is a consultant to Tuple Health, Inc. and was previously a part-time employee of Acumen, LLC. Dr. Weissman has nothing to disclose.

Automated early warning systems (EWSs) use data inputs to recognize clinical states requiring time-sensitive intervention and then generate notifications through different modalities to clinicians. EWSs serve as common tools for improving the recognition and treatment of important clinical states such as sepsis. However, despite the early enthusiasm, these warning systems have often yielded disappointing outcomes. In sepsis, for example, EWSs have shown mixed results in clinical trials, and concerns regarding the overuse of EWSs in diagnosing sepsis have grown.^1-4 We argue that inattention to the importance of timing in EWS training and evaluation provides one reason that EWSs have underperformed. Thus, to improve care, a warning system must not only identify the clinical state accurately, but it must also do so in a sufficiently timely manner to implement the associated interventions, such as administration of antibiotics for sepsis. Although the literature has occasionally highlighted the importance of timing in electronic surveillance systems, no one has linked the temporal dependence of performance metrics and intervention feasibility to the failure of such warning systems and explained how to operationalize timing in their development.^5-8 Using sepsis as an example, we explain why timing is important and propose new metrics and strategies for training and evaluating EWS models. EWSs are divided into two types: detection systems that recognize critical illnesses at a particular moment and prediction systems that estimate risk of deterioration over varying time frames.⁹ We focus primarily on detection systems, but our analysis is also important for prediction systems, which we will discuss in the last section.

EWS metrics have evolved from focusing on crude measures of discrimination to more clinically relevant metrics, such as the positive predictive value (PPV). The common performance metrics, including the c-statistic, evaluate the performance of EWSs in distinguishing events from nonevents, such as the presence or absence of sepsis in hospitalized patients. However, the c-statistic does not account for disease prevalence. A given c-statistic is compatible with a wide range of PPVs; a low PPV may limit an EWS’s usefulness to promote interventions and generate increased alert fatigue.¹⁰

However, the PPV, although important, provides no information on the timing of state recognition in relation to clinical time zero. Time zero is the first moment at which a critical state can be recognized based on available data and current medical science. Different approaches, including laboratory values, clinical assessments, retrospective chart reviews, triage times, and others, have been used to measure time zero.^8,11-13 All these approaches feature advantages and disadvantages; the evaluation of timing will exhibit sensitivity to the approach used.¹⁴ Further work is needed to gain additional insights into the measurement of time zero.

a) 50% true cases of sepsis, with a mean time of 35 minutes after clinical time zero;

b) 50% true cases, with a mean time of 60 minutes before clinical time zero (prediction EWS);

c) 50% true cases of sepsis, with a mean time of 1.3 days since clinical time zero, but with 70% of these cases undiagnosed at the time of EWS detection;

d) 50% true cases of cases, with mean time of 1.3 days since clinical time zero, that is, all cases among those promptly detected and treated through routine clinician oversight.

Each of these situations features differing clinical utility to help meet the hospital objective of increasing early administration of antibiotics. More generally, three dimensions of timing are important for detection systems. The first dimension is the timing of detection relative to time zero. The second is the timing relative to ”real-world” clinician detection. The third is timing with respect to the associated clinical objective. For a given PPV, an EWS performs better when detecting a state (1) at, near, or in advance of time zero, (2) prior to clinician detection, and (3) sufficiently in advance of an operational objective to promote change. On the other hand, when an EWS consistently sends alerts after clinician action, it serves a lesser purpose and risks causing alert fatigue; such cases have been described in studies.¹⁵

Acknowledging the importance of timing features implications for researchers and health system leaders. Researchers who develop EWS should include how these systems perform relative to both time zero and critical milestones in the clinical course. Operational leadership should understand the trade-offs that occur between alert fatigue (through lower PPV at the margin with earlier detection) and lead time to implement an intervention. Navigating these trade-offs involves a complex organizational decision. The “number needed to evaluate” is one way to quantify this fatigue factor.¹⁶ Such a measure gives a sense of the number of cases a clinician will need to evaluate per event. Collaborations between clinical leadership, operational leadership, and data scientists are needed to determine how to evaluate individual systems.

A good metric should capture the three important dimensions of timing while retaining intuitiveness to clinicians and leadership. One graphical option involves plotting the PPVs over time and relative to the clinical state evolution (Figure). This PPV-over-time curve shows when true positives occur relative to the time course of sepsis, including the three major dimensions of timing. This curve can also show a “clinically important window (CIW)”, which is bounded on the right by the latest point in time when recognition could still meet the clinical objective. For sepsis, the curve might be bounded at 2.5 hours to meet an objective of antibiotics within three hours, with the assumption that 0.5 hour is needed for a response. For detection systems, the window would be bounded on the left by clinical time zero. The graph can also designate the point when most cases of sepsis have been recognized clinically with historical data. The Figure depicts an example curve for a detection model.

The metrics derived from this curve may be used alongside the PPV for training and evaluation. Often, adjusting the PPV for its relationship to time zero and the CIW will aid in recognizing the existence of a time beyond which detection fails to help achieve the intended intervention. Detection beyond the window should not credited as a true positive if it fails to facilitate the objective. One option is to credit detection at or before time zero as one and discount later detection by the delay from time zero. More specifically, a true positive could be discounted by the difference between the end of the CIW and the moment of detection divided by the CIW length. This discounted PPV could be displayed alongside the PPV to gauge the temporal dimension of performance and be used for training.

The use of timing places additional demands on validation owing to the need for a time-based gold standard. In such a case, the unit of analysis in system development might not be the patient encounter but rather the patient-hour or patient-15-minute epoch, depending on how frequently the EWS updates risk information and may alert. By contrast, the sepsis detection models used in administrative databases rely on an encounter-level PPV, which provides more limited information compared with real-time EWSs.¹⁷ When time zero cannot be measured, alternatives may be used to capture several dimensions of timing; these alternatives include measurement of the percentage of cases that recognize the event prior to clinicians.¹⁵

Detection systems face the limitation that they lack the capability to identify a state before its occurrence. Prediction systems are more likely to be actionable, as they provide more lead time for intervention, but accurate prediction models are also more difficult to develop. With a predictive system, an additional dimension of timing becomes important: the time horizon for prediction. Prediction models may be trained to recognize a state within a specific time frame (eg, 6, 12, or 24 hours), and test characteristics, including PPV, may vary with the window.¹⁸ A given PPV (of eventual development of sepsis) is compatible with varying time windows and thus again lacks important information on performance.

The timing relative to clinical time zero remains important for prediction. For a predictive EWS, the graph in the figure may be expected to shift to the left. Models with good performance will occasionally send an alert after time zero. For a prediction system with a time horizon of six hours, it is more useful to have alerts occur a mean time of four hours prior to time zero than four minutes prior.

Improving the clinical utility of EWSs requires better measurement of timing. Researchers should incorporate timing into system development, and operational leaders should be cognizant of timing during implementation. Specific steps should include devising better strategies to estimate the relationship of state recognition to clinical time zero and developing methods to discount recognition when it occurs too late to be actionable.

Disclosures

Dr. Rolnick is a consultant to Tuple Health, Inc. and was previously a part-time employee of Acumen, LLC. Dr. Weissman has nothing to disclose.