It is an uncomfortable and unavoidable reality as physicians that for every procedure we learn, there must be a first time we perform it. As with any type of skill, it takes practice to become proficient. The unique challenge in medicine is that the practice involves performing procedures on real patients. We cannot avoid the hands-on nature of the training process; we can, however, approach its ethical challenges mindfully. Herein, I will discuss some of the ethical considerations in providing care as a trainee and identify potential barriers to best practices, particularly as they relate to procedural dermatology.

Tell Patients You Are in Training

In every patient encounter, we must introduce ourselves as a trainee. The principle of right to the truth dictates that we are transparent about our level of training and do not misrepresent ourselves to our patients. A statement released by the American Medical Association (AMA) Council on Ethical and Judicial Affairs asserts that “[p]atients should be informed of the identity and training status of individuals involved in their care.”¹

Although straightforward in theory, this mandate is not always simple in practice. With patients unfamiliar with the health care system, it could be more onerous to clearly communicate training status than simply introducing oneself as a resident. A study conducted in the emergency department at Vanderbilt University Hospital (Nashville, Tennessee) found that many patients and their family members (N=430) did not understand the various roles and responsibilities of physicians in the teaching hospital setting. For example, 30% believed an attending physician requires supervision by a resident, and an additional 17% of those surveyed were not sure.² The AMA requests we “refrain from using terms that may be confusing when describing the training status of the students,”¹ which evidently is audience specific. Thus, as with any type of patient education, a thorough introduction may require assessment of understanding.

Disclosure of Experience Level With a Particular Procedure

There is a clear professional expectation that we disclose to patients that we are in training; however, a universal standard does not exist for disclosure of our exact level of experience in a particular procedure. Do we need to tell patients if it is our first time performing a given procedure? What if it is our tenth? Multiple studies have found that patients want specifics. In one study of bariatric surgery patients (N=108), 93% felt that they should always be informed if it was the first time a trainee was performing a particular procedure.³ A study conducted in the emergency department setting (N=202) also found that the majority of patients thought they should be informed if a resident was performing a procedure for the first time, but the distribution differed by procedure (66% for suturing vs 82% for lumbar puncture).⁴

Despite these findings, this degree of specificity is not always discussed with patients and perhaps does not need to be. LaRosa and Grant-Kels⁵ analyzed a hypothetical scenario in which a dermatology resident is to perform his first excision under attending supervision and concluded that broad disclosure of training status would suffice in the given scenario, as it would not be necessary to state that it was his first time performing an excision. It is unclear if the same conclusion could be drawn for all procedures and levels of experience. Outcome data would help inform the analysis, but the available data are from other specialties including general surgery, gynecology, and urology. Some studies demonstrate an increased risk of adverse outcomes with trainee involvement in procedures such as bariatric surgery and emergency general surgery, but the data are mixed and may not be generalizable to dermatologic procedures.^6-8

The appropriate level of detail to disclose regarding a physician’s experience may need to be assessed on a case-by-case basis, and the principles of informed consent can help. Informed consent requires understanding of the diagnosis, the treatment options including nonintervention, and the risks and benefits of each alternative. In obtaining informed consent, we must disclose “any facts which are necessary to form the basis of an intelligent consent by the patient to the proposed treatment.”⁹ Providers must determine what aspects of a trainee’s experience level are relevant to the risk-benefit analysis in a given set of circumstances. Surely, there is a large degree of subjectivity in this determination as data are limited, but information deemed relevant must be shared. Information that is inconsequential, on the other hand, may be omitted. It could even be argued that more detailed information, especially if it may cause anxiety, would be detrimental to share. For example, we would not list the chemical name of every preservative in every vaccine we recommend for children if there is no evidence of inflicting harm. If the information has not been shown to have clinical impact or affect safety concerns, the anxiety may be undue.

Withholding Information Can Violate Ethical Principles

We must be careful not to withhold details of our experience level with a particular procedure for the wrong reasons. It would be wrong, for example, to withhold information simply to avoid causing anxiety, which could be seen as an invocation of therapeutic privilege, a controversial practice of withholding important information that poses a psychological threat to the patient. A classic example is the physician who defers disclosure of a terminal diagnosis to preserve hope. Although therapeutic privilege theoretically promotes the principle of beneficence, it violates the principles of autonomy and right to truth and therefore generally is regarded as unethically paternalistic in modern medical ethics.⁹

Patients Can Refuse Trainee Participation

It also is unethical to withhold information to obtain consent and avoid refusal of our care. Refusal of trainee participation is not uncommon. In the aforementioned study of bariatric surgery patients, 92.4% supported their procedure being performed at a teaching hospital, but only 56% would consent to a resident assisting staff during the procedure. A mere 33% of those patients would consent to a resident primarily performing with staff assisting.³ Although the proportion of patients who refuse certainly depends on the type of procedure among other factors, it is a reality in any teaching environment. The training paradigm in medicine depends on being able to practice procedures with supervision before we are independent providers. If patients refuse our care, our training suffers. However, the AMA maintains that “[p]atients are free to choose from whom they receive treatment,”¹ and we must respect this aspect of patient autonomy.

Final Thoughts

When it comes to the performance of procedures, there are a few basic principles to keep in mind to provide ethical care to our patients while we are in training. Although we must accept that a crucial part of learning dermatologic procedures is hands on with real patients, we also need to come prepared having learned what we can through reading and practice with cadavers or skin substitutes. Procedures we execute as residents should be performed with adequate supervision, and as we progress through residency, we should be given increased autonomy and graded responsibility to prepare us for independent practice at graduation. Although it is the responsibility of the attending physician to provide appropriate oversight for the resident’s level of training, we should feel empowered to ask for help and have the humility to know when we need it.

It is an uncomfortable and unavoidable reality as physicians that for every procedure we learn, there must be a first time we perform it. As with any type of skill, it takes practice to become proficient. The unique challenge in medicine is that the practice involves performing procedures on real patients. We cannot avoid the hands-on nature of the training process; we can, however, approach its ethical challenges mindfully. Herein, I will discuss some of the ethical considerations in providing care as a trainee and identify potential barriers to best practices, particularly as they relate to procedural dermatology.

Tell Patients You Are in Training

In every patient encounter, we must introduce ourselves as a trainee. The principle of right to the truth dictates that we are transparent about our level of training and do not misrepresent ourselves to our patients. A statement released by the American Medical Association (AMA) Council on Ethical and Judicial Affairs asserts that “[p]atients should be informed of the identity and training status of individuals involved in their care.”¹

Although straightforward in theory, this mandate is not always simple in practice. With patients unfamiliar with the health care system, it could be more onerous to clearly communicate training status than simply introducing oneself as a resident. A study conducted in the emergency department at Vanderbilt University Hospital (Nashville, Tennessee) found that many patients and their family members (N=430) did not understand the various roles and responsibilities of physicians in the teaching hospital setting. For example, 30% believed an attending physician requires supervision by a resident, and an additional 17% of those surveyed were not sure.² The AMA requests we “refrain from using terms that may be confusing when describing the training status of the students,”¹ which evidently is audience specific. Thus, as with any type of patient education, a thorough introduction may require assessment of understanding.

Disclosure of Experience Level With a Particular Procedure

There is a clear professional expectation that we disclose to patients that we are in training; however, a universal standard does not exist for disclosure of our exact level of experience in a particular procedure. Do we need to tell patients if it is our first time performing a given procedure? What if it is our tenth? Multiple studies have found that patients want specifics. In one study of bariatric surgery patients (N=108), 93% felt that they should always be informed if it was the first time a trainee was performing a particular procedure.³ A study conducted in the emergency department setting (N=202) also found that the majority of patients thought they should be informed if a resident was performing a procedure for the first time, but the distribution differed by procedure (66% for suturing vs 82% for lumbar puncture).⁴

Despite these findings, this degree of specificity is not always discussed with patients and perhaps does not need to be. LaRosa and Grant-Kels⁵ analyzed a hypothetical scenario in which a dermatology resident is to perform his first excision under attending supervision and concluded that broad disclosure of training status would suffice in the given scenario, as it would not be necessary to state that it was his first time performing an excision. It is unclear if the same conclusion could be drawn for all procedures and levels of experience. Outcome data would help inform the analysis, but the available data are from other specialties including general surgery, gynecology, and urology. Some studies demonstrate an increased risk of adverse outcomes with trainee involvement in procedures such as bariatric surgery and emergency general surgery, but the data are mixed and may not be generalizable to dermatologic procedures.^6-8

The appropriate level of detail to disclose regarding a physician’s experience may need to be assessed on a case-by-case basis, and the principles of informed consent can help. Informed consent requires understanding of the diagnosis, the treatment options including nonintervention, and the risks and benefits of each alternative. In obtaining informed consent, we must disclose “any facts which are necessary to form the basis of an intelligent consent by the patient to the proposed treatment.”⁹ Providers must determine what aspects of a trainee’s experience level are relevant to the risk-benefit analysis in a given set of circumstances. Surely, there is a large degree of subjectivity in this determination as data are limited, but information deemed relevant must be shared. Information that is inconsequential, on the other hand, may be omitted. It could even be argued that more detailed information, especially if it may cause anxiety, would be detrimental to share. For example, we would not list the chemical name of every preservative in every vaccine we recommend for children if there is no evidence of inflicting harm. If the information has not been shown to have clinical impact or affect safety concerns, the anxiety may be undue.

Withholding Information Can Violate Ethical Principles

We must be careful not to withhold details of our experience level with a particular procedure for the wrong reasons. It would be wrong, for example, to withhold information simply to avoid causing anxiety, which could be seen as an invocation of therapeutic privilege, a controversial practice of withholding important information that poses a psychological threat to the patient. A classic example is the physician who defers disclosure of a terminal diagnosis to preserve hope. Although therapeutic privilege theoretically promotes the principle of beneficence, it violates the principles of autonomy and right to truth and therefore generally is regarded as unethically paternalistic in modern medical ethics.⁹

Patients Can Refuse Trainee Participation

It also is unethical to withhold information to obtain consent and avoid refusal of our care. Refusal of trainee participation is not uncommon. In the aforementioned study of bariatric surgery patients, 92.4% supported their procedure being performed at a teaching hospital, but only 56% would consent to a resident assisting staff during the procedure. A mere 33% of those patients would consent to a resident primarily performing with staff assisting.³ Although the proportion of patients who refuse certainly depends on the type of procedure among other factors, it is a reality in any teaching environment. The training paradigm in medicine depends on being able to practice procedures with supervision before we are independent providers. If patients refuse our care, our training suffers. However, the AMA maintains that “[p]atients are free to choose from whom they receive treatment,”¹ and we must respect this aspect of patient autonomy.

Final Thoughts

When it comes to the performance of procedures, there are a few basic principles to keep in mind to provide ethical care to our patients while we are in training. Although we must accept that a crucial part of learning dermatologic procedures is hands on with real patients, we also need to come prepared having learned what we can through reading and practice with cadavers or skin substitutes. Procedures we execute as residents should be performed with adequate supervision, and as we progress through residency, we should be given increased autonomy and graded responsibility to prepare us for independent practice at graduation. Although it is the responsibility of the attending physician to provide appropriate oversight for the resident’s level of training, we should feel empowered to ask for help and have the humility to know when we need it.

It is an uncomfortable and unavoidable reality as physicians that for every procedure we learn, there must be a first time we perform it. As with any type of skill, it takes practice to become proficient. The unique challenge in medicine is that the practice involves performing procedures on real patients. We cannot avoid the hands-on nature of the training process; we can, however, approach its ethical challenges mindfully. Herein, I will discuss some of the ethical considerations in providing care as a trainee and identify potential barriers to best practices, particularly as they relate to procedural dermatology.

Tell Patients You Are in Training

In every patient encounter, we must introduce ourselves as a trainee. The principle of right to the truth dictates that we are transparent about our level of training and do not misrepresent ourselves to our patients. A statement released by the American Medical Association (AMA) Council on Ethical and Judicial Affairs asserts that “[p]atients should be informed of the identity and training status of individuals involved in their care.”¹

Although straightforward in theory, this mandate is not always simple in practice. With patients unfamiliar with the health care system, it could be more onerous to clearly communicate training status than simply introducing oneself as a resident. A study conducted in the emergency department at Vanderbilt University Hospital (Nashville, Tennessee) found that many patients and their family members (N=430) did not understand the various roles and responsibilities of physicians in the teaching hospital setting. For example, 30% believed an attending physician requires supervision by a resident, and an additional 17% of those surveyed were not sure.² The AMA requests we “refrain from using terms that may be confusing when describing the training status of the students,”¹ which evidently is audience specific. Thus, as with any type of patient education, a thorough introduction may require assessment of understanding.

Disclosure of Experience Level With a Particular Procedure

There is a clear professional expectation that we disclose to patients that we are in training; however, a universal standard does not exist for disclosure of our exact level of experience in a particular procedure. Do we need to tell patients if it is our first time performing a given procedure? What if it is our tenth? Multiple studies have found that patients want specifics. In one study of bariatric surgery patients (N=108), 93% felt that they should always be informed if it was the first time a trainee was performing a particular procedure.³ A study conducted in the emergency department setting (N=202) also found that the majority of patients thought they should be informed if a resident was performing a procedure for the first time, but the distribution differed by procedure (66% for suturing vs 82% for lumbar puncture).⁴

Despite these findings, this degree of specificity is not always discussed with patients and perhaps does not need to be. LaRosa and Grant-Kels⁵ analyzed a hypothetical scenario in which a dermatology resident is to perform his first excision under attending supervision and concluded that broad disclosure of training status would suffice in the given scenario, as it would not be necessary to state that it was his first time performing an excision. It is unclear if the same conclusion could be drawn for all procedures and levels of experience. Outcome data would help inform the analysis, but the available data are from other specialties including general surgery, gynecology, and urology. Some studies demonstrate an increased risk of adverse outcomes with trainee involvement in procedures such as bariatric surgery and emergency general surgery, but the data are mixed and may not be generalizable to dermatologic procedures.^6-8

The appropriate level of detail to disclose regarding a physician’s experience may need to be assessed on a case-by-case basis, and the principles of informed consent can help. Informed consent requires understanding of the diagnosis, the treatment options including nonintervention, and the risks and benefits of each alternative. In obtaining informed consent, we must disclose “any facts which are necessary to form the basis of an intelligent consent by the patient to the proposed treatment.”⁹ Providers must determine what aspects of a trainee’s experience level are relevant to the risk-benefit analysis in a given set of circumstances. Surely, there is a large degree of subjectivity in this determination as data are limited, but information deemed relevant must be shared. Information that is inconsequential, on the other hand, may be omitted. It could even be argued that more detailed information, especially if it may cause anxiety, would be detrimental to share. For example, we would not list the chemical name of every preservative in every vaccine we recommend for children if there is no evidence of inflicting harm. If the information has not been shown to have clinical impact or affect safety concerns, the anxiety may be undue.

Withholding Information Can Violate Ethical Principles

We must be careful not to withhold details of our experience level with a particular procedure for the wrong reasons. It would be wrong, for example, to withhold information simply to avoid causing anxiety, which could be seen as an invocation of therapeutic privilege, a controversial practice of withholding important information that poses a psychological threat to the patient. A classic example is the physician who defers disclosure of a terminal diagnosis to preserve hope. Although therapeutic privilege theoretically promotes the principle of beneficence, it violates the principles of autonomy and right to truth and therefore generally is regarded as unethically paternalistic in modern medical ethics.⁹

Patients Can Refuse Trainee Participation

It also is unethical to withhold information to obtain consent and avoid refusal of our care. Refusal of trainee participation is not uncommon. In the aforementioned study of bariatric surgery patients, 92.4% supported their procedure being performed at a teaching hospital, but only 56% would consent to a resident assisting staff during the procedure. A mere 33% of those patients would consent to a resident primarily performing with staff assisting.³ Although the proportion of patients who refuse certainly depends on the type of procedure among other factors, it is a reality in any teaching environment. The training paradigm in medicine depends on being able to practice procedures with supervision before we are independent providers. If patients refuse our care, our training suffers. However, the AMA maintains that “[p]atients are free to choose from whom they receive treatment,”¹ and we must respect this aspect of patient autonomy.

Final Thoughts

When it comes to the performance of procedures, there are a few basic principles to keep in mind to provide ethical care to our patients while we are in training. Although we must accept that a crucial part of learning dermatologic procedures is hands on with real patients, we also need to come prepared having learned what we can through reading and practice with cadavers or skin substitutes. Procedures we execute as residents should be performed with adequate supervision, and as we progress through residency, we should be given increased autonomy and graded responsibility to prepare us for independent practice at graduation. Although it is the responsibility of the attending physician to provide appropriate oversight for the resident’s level of training, we should feel empowered to ask for help and have the humility to know when we need it.

SAN DIEGO – For patients with acute gastroenteritis, multiplex polymerase chain reaction (PCR)–based gastrointestinal pathogen testing was associated with lower resource utilization and less antibiotic prescribing, compared with conventional stool culture methods. However, antibiotics were still prescribed for more than one in three patients tested by any method.

“A positive test by any modality did result in decreased utilization of endoscopy, radiology, and antibiotic prescribing, but this effect appeared to be much greater for the GI PCR assay,” said Jordan Axelrad, MD, speaking at the annual Digestive Disease Week.

“Overall, patients who received GI PCR were 12% less likely to undergo endoscopy, 7% less likely to undergo abdominal radiography, and 11% less likely to be prescribed any antibiotic,” compared with patients who were tested by conventional stool culture, said Dr. Axelrad, a gastroenterologist at New York University.

In a cross-sectional study, Dr. Axelrad and his coauthors looked at patients who underwent stool testing for the 26 months before (n = 5,986) and after (n = 9,402) March 2015, when Dr. Axelrad’s home institution switched from conventional stool culture to the GI PCR panel. For the earlier time period, the investigators included patients who received stool culture both with and without an ova and parasites exam, as well as those who underwent enzyme-linked immunosorbent assay viral testing for rotavirus and adenovirus.

Patient demographic data were included as study variables; additionally, the study tracked utilization of endoscopy, abdominal, or other radiology studies, and ED visits for 30 days after testing. They also included any antibiotic prescribing within the 14 days post testing.

Roughly one-third of patients were tested as outpatients, 1 in 10 in the ED, and the remainder as inpatients. Patient age was a mean 46.7 years for the culture group, and 45.5 years for the GI PCR group.

The multiplex PCR test used in the study tested for 12 gastrointestinal pathogenic bacteria, 4 parasites, and 5 viruses.

As expected, PCR testing yielded a higher positive test rate than conventional stool testing, even when EIA tests were included (29.2% vs. 4.1%). In the 2,746 patients with a positive GI PCR test, a total of 3,804 pathogens were identified. Adenovirus accounted for 39% of these positive results. Positive bacterial results were seen in about 65.0% of the positive subgroup, with Escherichia coli subtypes seen in 51.7% of the positive tests.

Overall, positive results for viruses, bacteria, and multiple pathogens were more likely with GI PCR testing, compared with conventional testing (P = .001 for all). Parasites accounted for only 8.2% of the positive PCR test results, but this was significantly more than the 3.7% seen with conventional testing (P = .011).

At the 14-day mark post testing, “Patients who underwent a GI panel were less likely to be prescribed any antibiotic. But overall, antibiotics were fairly common in both groups,” said Dr. Axelrad, noting that 41% of patients who underwent stool culture received an antibiotic by 14 days, compared with 36% for patients who underwent a GI PCR panel (P = .001).

By the end of 30 days, most patients in each group had not received an endoscopic procedure, with significantly more procedure-free patients in the PCR group (91.6% vs. 90.4%; P = .008).

Against a backdrop of slightly higher overall radiology utilization in the PCR group – potentially attributable to practice trends over time – abdominal radiology was less likely for these patients than for the culture group (11.4% vs. 12.8%; P = .011).

The 30-day ED visit rate was low and similar between groups (11.4% for PCR vs. 12.8% for culture; P = .116).

The much quicker turnaround for the GI PCR panel didn’t translate into a shorter length of stay, though: Inpatient length of stay was a median 5 days in both groups.

“We feel that the outcomes that we noted were likely due to the increased sensitivity and specificity” of the PCR-based testing, said Dr. Axelrad. “Obviously, if you have more pathogen-positive findings, you may be less likely to order extensive testing. And if you’ve identified something like norovirus, you may feel reassured, and not order further testing.”

Dr. Axelrad pointed out that his institution’s overall PCR positivity rates were lower than the 70% rates some other studies have reported. “We feel that, given our large sample size, our results may more accurately reflect clinical practice, and perhaps that lower positivity rate may reflect increased use of this test in an inpatient setting,” he said. “We’re looking at that.”

Study limitations included the retrospective nature of the study. “Also, as we all know, PCR testing fails to discriminate between active infection and asymptomatic colonization,” raising questions about whether a positive PCR test really indicates true infection, noted Dr. Axelrad.

“Coupled with a high-sensitivity rapid turnaround, there’s the potential to reduce costs, but the cost-effectiveness of these assays has not been fully determined. There are several studies looking at this,” with results still to come, he said.

The notable reduction in antibiotic prescribing for those patients who received PCR-based testing means that GI PCR panels could be a useful tool to promote antibiotic stewardship, though Dr. Axelrad also noted that “antibiotics were still used in about a third of all patients.”

Dr. Axelrad reported no outside sources of funding. He has performed consulting services for and received research funding from BioFire, which manufactured the GI PCR assay used in the study, but BioFire did not fund this research.

[email protected]

SOURCE: Axelrad J et al. DDW 2019, Presentation 978.

SAN DIEGO – For patients with acute gastroenteritis, multiplex polymerase chain reaction (PCR)–based gastrointestinal pathogen testing was associated with lower resource utilization and less antibiotic prescribing, compared with conventional stool culture methods. However, antibiotics were still prescribed for more than one in three patients tested by any method.

“A positive test by any modality did result in decreased utilization of endoscopy, radiology, and antibiotic prescribing, but this effect appeared to be much greater for the GI PCR assay,” said Jordan Axelrad, MD, speaking at the annual Digestive Disease Week.

“Overall, patients who received GI PCR were 12% less likely to undergo endoscopy, 7% less likely to undergo abdominal radiography, and 11% less likely to be prescribed any antibiotic,” compared with patients who were tested by conventional stool culture, said Dr. Axelrad, a gastroenterologist at New York University.

In a cross-sectional study, Dr. Axelrad and his coauthors looked at patients who underwent stool testing for the 26 months before (n = 5,986) and after (n = 9,402) March 2015, when Dr. Axelrad’s home institution switched from conventional stool culture to the GI PCR panel. For the earlier time period, the investigators included patients who received stool culture both with and without an ova and parasites exam, as well as those who underwent enzyme-linked immunosorbent assay viral testing for rotavirus and adenovirus.

Patient demographic data were included as study variables; additionally, the study tracked utilization of endoscopy, abdominal, or other radiology studies, and ED visits for 30 days after testing. They also included any antibiotic prescribing within the 14 days post testing.

Roughly one-third of patients were tested as outpatients, 1 in 10 in the ED, and the remainder as inpatients. Patient age was a mean 46.7 years for the culture group, and 45.5 years for the GI PCR group.

The multiplex PCR test used in the study tested for 12 gastrointestinal pathogenic bacteria, 4 parasites, and 5 viruses.

As expected, PCR testing yielded a higher positive test rate than conventional stool testing, even when EIA tests were included (29.2% vs. 4.1%). In the 2,746 patients with a positive GI PCR test, a total of 3,804 pathogens were identified. Adenovirus accounted for 39% of these positive results. Positive bacterial results were seen in about 65.0% of the positive subgroup, with Escherichia coli subtypes seen in 51.7% of the positive tests.

Overall, positive results for viruses, bacteria, and multiple pathogens were more likely with GI PCR testing, compared with conventional testing (P = .001 for all). Parasites accounted for only 8.2% of the positive PCR test results, but this was significantly more than the 3.7% seen with conventional testing (P = .011).

At the 14-day mark post testing, “Patients who underwent a GI panel were less likely to be prescribed any antibiotic. But overall, antibiotics were fairly common in both groups,” said Dr. Axelrad, noting that 41% of patients who underwent stool culture received an antibiotic by 14 days, compared with 36% for patients who underwent a GI PCR panel (P = .001).

By the end of 30 days, most patients in each group had not received an endoscopic procedure, with significantly more procedure-free patients in the PCR group (91.6% vs. 90.4%; P = .008).

Against a backdrop of slightly higher overall radiology utilization in the PCR group – potentially attributable to practice trends over time – abdominal radiology was less likely for these patients than for the culture group (11.4% vs. 12.8%; P = .011).

The 30-day ED visit rate was low and similar between groups (11.4% for PCR vs. 12.8% for culture; P = .116).

The much quicker turnaround for the GI PCR panel didn’t translate into a shorter length of stay, though: Inpatient length of stay was a median 5 days in both groups.

“We feel that the outcomes that we noted were likely due to the increased sensitivity and specificity” of the PCR-based testing, said Dr. Axelrad. “Obviously, if you have more pathogen-positive findings, you may be less likely to order extensive testing. And if you’ve identified something like norovirus, you may feel reassured, and not order further testing.”

Dr. Axelrad pointed out that his institution’s overall PCR positivity rates were lower than the 70% rates some other studies have reported. “We feel that, given our large sample size, our results may more accurately reflect clinical practice, and perhaps that lower positivity rate may reflect increased use of this test in an inpatient setting,” he said. “We’re looking at that.”

Study limitations included the retrospective nature of the study. “Also, as we all know, PCR testing fails to discriminate between active infection and asymptomatic colonization,” raising questions about whether a positive PCR test really indicates true infection, noted Dr. Axelrad.

“Coupled with a high-sensitivity rapid turnaround, there’s the potential to reduce costs, but the cost-effectiveness of these assays has not been fully determined. There are several studies looking at this,” with results still to come, he said.

The notable reduction in antibiotic prescribing for those patients who received PCR-based testing means that GI PCR panels could be a useful tool to promote antibiotic stewardship, though Dr. Axelrad also noted that “antibiotics were still used in about a third of all patients.”

Dr. Axelrad reported no outside sources of funding. He has performed consulting services for and received research funding from BioFire, which manufactured the GI PCR assay used in the study, but BioFire did not fund this research.

[email protected]

SOURCE: Axelrad J et al. DDW 2019, Presentation 978.

SAN DIEGO – For patients with acute gastroenteritis, multiplex polymerase chain reaction (PCR)–based gastrointestinal pathogen testing was associated with lower resource utilization and less antibiotic prescribing, compared with conventional stool culture methods. However, antibiotics were still prescribed for more than one in three patients tested by any method.

“A positive test by any modality did result in decreased utilization of endoscopy, radiology, and antibiotic prescribing, but this effect appeared to be much greater for the GI PCR assay,” said Jordan Axelrad, MD, speaking at the annual Digestive Disease Week.

“Overall, patients who received GI PCR were 12% less likely to undergo endoscopy, 7% less likely to undergo abdominal radiography, and 11% less likely to be prescribed any antibiotic,” compared with patients who were tested by conventional stool culture, said Dr. Axelrad, a gastroenterologist at New York University.

In a cross-sectional study, Dr. Axelrad and his coauthors looked at patients who underwent stool testing for the 26 months before (n = 5,986) and after (n = 9,402) March 2015, when Dr. Axelrad’s home institution switched from conventional stool culture to the GI PCR panel. For the earlier time period, the investigators included patients who received stool culture both with and without an ova and parasites exam, as well as those who underwent enzyme-linked immunosorbent assay viral testing for rotavirus and adenovirus.

Patient demographic data were included as study variables; additionally, the study tracked utilization of endoscopy, abdominal, or other radiology studies, and ED visits for 30 days after testing. They also included any antibiotic prescribing within the 14 days post testing.

Roughly one-third of patients were tested as outpatients, 1 in 10 in the ED, and the remainder as inpatients. Patient age was a mean 46.7 years for the culture group, and 45.5 years for the GI PCR group.

The multiplex PCR test used in the study tested for 12 gastrointestinal pathogenic bacteria, 4 parasites, and 5 viruses.

As expected, PCR testing yielded a higher positive test rate than conventional stool testing, even when EIA tests were included (29.2% vs. 4.1%). In the 2,746 patients with a positive GI PCR test, a total of 3,804 pathogens were identified. Adenovirus accounted for 39% of these positive results. Positive bacterial results were seen in about 65.0% of the positive subgroup, with Escherichia coli subtypes seen in 51.7% of the positive tests.

Overall, positive results for viruses, bacteria, and multiple pathogens were more likely with GI PCR testing, compared with conventional testing (P = .001 for all). Parasites accounted for only 8.2% of the positive PCR test results, but this was significantly more than the 3.7% seen with conventional testing (P = .011).

At the 14-day mark post testing, “Patients who underwent a GI panel were less likely to be prescribed any antibiotic. But overall, antibiotics were fairly common in both groups,” said Dr. Axelrad, noting that 41% of patients who underwent stool culture received an antibiotic by 14 days, compared with 36% for patients who underwent a GI PCR panel (P = .001).

By the end of 30 days, most patients in each group had not received an endoscopic procedure, with significantly more procedure-free patients in the PCR group (91.6% vs. 90.4%; P = .008).

Against a backdrop of slightly higher overall radiology utilization in the PCR group – potentially attributable to practice trends over time – abdominal radiology was less likely for these patients than for the culture group (11.4% vs. 12.8%; P = .011).

The 30-day ED visit rate was low and similar between groups (11.4% for PCR vs. 12.8% for culture; P = .116).

The much quicker turnaround for the GI PCR panel didn’t translate into a shorter length of stay, though: Inpatient length of stay was a median 5 days in both groups.

“We feel that the outcomes that we noted were likely due to the increased sensitivity and specificity” of the PCR-based testing, said Dr. Axelrad. “Obviously, if you have more pathogen-positive findings, you may be less likely to order extensive testing. And if you’ve identified something like norovirus, you may feel reassured, and not order further testing.”

Dr. Axelrad pointed out that his institution’s overall PCR positivity rates were lower than the 70% rates some other studies have reported. “We feel that, given our large sample size, our results may more accurately reflect clinical practice, and perhaps that lower positivity rate may reflect increased use of this test in an inpatient setting,” he said. “We’re looking at that.”

Study limitations included the retrospective nature of the study. “Also, as we all know, PCR testing fails to discriminate between active infection and asymptomatic colonization,” raising questions about whether a positive PCR test really indicates true infection, noted Dr. Axelrad.

“Coupled with a high-sensitivity rapid turnaround, there’s the potential to reduce costs, but the cost-effectiveness of these assays has not been fully determined. There are several studies looking at this,” with results still to come, he said.

The notable reduction in antibiotic prescribing for those patients who received PCR-based testing means that GI PCR panels could be a useful tool to promote antibiotic stewardship, though Dr. Axelrad also noted that “antibiotics were still used in about a third of all patients.”

Dr. Axelrad reported no outside sources of funding. He has performed consulting services for and received research funding from BioFire, which manufactured the GI PCR assay used in the study, but BioFire did not fund this research.

[email protected]

SOURCE: Axelrad J et al. DDW 2019, Presentation 978.

Pre-exposure prophylaxis (PrEP) plus effective antiretroviral therapy should be offered to people at high risk of HIV acquisition, according to a new recommendation from the U.S. Preventive Services Task Force (USPSTF).

“The USPSTF concludes with high certainty that the net benefit of the use of PrEP to reduce the risk of acquisition of HIV infection in persons at high risk of HIV infection is substantial,” wrote first author Douglas K. Owens, MD, of Stanford (Calif.) University and fellow members of the USPSTF. The recommendation was published in JAMA.

In various at-risk groups – including men who have sex with men, people at risk through heterosexual contact, and people who inject drugs – the USPSTF recommends a Food and Drug Adminstration–approved, once-daily oral treatment with combined tenofovir disoproxil fumarate and emtricitabine. In addition, tenofovir disoproxil fumarate alone can be considered an alternative regimen for high-risk heterosexually active men and women and people who inject drugs.

This recommendation was developed after a systematic review of PrEP’s effects on HIV, adherence to the treatment, and accuracy in identifying potential treatment candidates. “The findings of this review are generally consistent with those from other recent meta-analyses that found PrEP to be effective at reducing risk of HIV infection and found greater effectiveness in trials reporting higher adherence,” wrote Roger Chou, MD, of Oregon Health & Science University in Portland and coauthors. Their study was also published in JAMA.

To comprehensively assess PrEP and thus inform the USPSTF’s HIV prevention recommendations, the researchers reviewed criteria-meeting studies on oral PrEP with tenofovir disoproxil fumarate/emtricitabine or tenofovir disoproxil fumarate monotherapy; on the diagnostic accuracy of instruments to predict HIV infection; and on PrEP adherence. The final analysis included 14 randomized clinical trials, 8 observational studies, and 7 studies of diagnostic accuracy.

In 11 of the trials, PrEP was associated with reduced risk of HIV infection versus placebo or no PrEP (relative risk, 0.46; 95% confidence interval, 0.33-0.66). In 6 trials with adherence 70% or greater, the relative risk was 0.27 (95% CI, 0.19-0.39). In 7 studies on risk assessment tools for HIV infection, the instruments had moderate discrimination, though several of the studies had methodological shortcomings. As for serious adverse events, an analysis of 12 trials found no significant difference between PrEP and placebo (RR, 0.93; 95% CI, 0.77-1.12).

Dr. Chou and coauthors noted their study’s limitations, including analyzing English-language articles only and the random-effects model used to pool studies potentially returning narrow CIs. They did note, however, that the “analyses were repeated using the profile likelihood method,” which produced similar findings.

All members of the USPSTF receive travel reimbursement and an honorarium for participating in meetings. The study was funded by the Department of Health and Human Services. One of the authors reported receiving grants from the National Institutes of Health/National Institute on Drug Abuse and serving as principal investigator of NIH-funded clinical trials that received donated drugs from two pharmaceutical companies. No other conflicts of interest were reported.

SOURCE: Owens DK et al. JAMA. 2019 Jun 11. doi: 10.1001/jama.2019.6390; Chou R et al. JAMA. 2019 Jun 11. doi: 10.1001/jama.2019.2591.

Pre-exposure prophylaxis (PrEP) plus effective antiretroviral therapy should be offered to people at high risk of HIV acquisition, according to a new recommendation from the U.S. Preventive Services Task Force (USPSTF).

“The USPSTF concludes with high certainty that the net benefit of the use of PrEP to reduce the risk of acquisition of HIV infection in persons at high risk of HIV infection is substantial,” wrote first author Douglas K. Owens, MD, of Stanford (Calif.) University and fellow members of the USPSTF. The recommendation was published in JAMA.

In various at-risk groups – including men who have sex with men, people at risk through heterosexual contact, and people who inject drugs – the USPSTF recommends a Food and Drug Adminstration–approved, once-daily oral treatment with combined tenofovir disoproxil fumarate and emtricitabine. In addition, tenofovir disoproxil fumarate alone can be considered an alternative regimen for high-risk heterosexually active men and women and people who inject drugs.

This recommendation was developed after a systematic review of PrEP’s effects on HIV, adherence to the treatment, and accuracy in identifying potential treatment candidates. “The findings of this review are generally consistent with those from other recent meta-analyses that found PrEP to be effective at reducing risk of HIV infection and found greater effectiveness in trials reporting higher adherence,” wrote Roger Chou, MD, of Oregon Health & Science University in Portland and coauthors. Their study was also published in JAMA.

To comprehensively assess PrEP and thus inform the USPSTF’s HIV prevention recommendations, the researchers reviewed criteria-meeting studies on oral PrEP with tenofovir disoproxil fumarate/emtricitabine or tenofovir disoproxil fumarate monotherapy; on the diagnostic accuracy of instruments to predict HIV infection; and on PrEP adherence. The final analysis included 14 randomized clinical trials, 8 observational studies, and 7 studies of diagnostic accuracy.

In 11 of the trials, PrEP was associated with reduced risk of HIV infection versus placebo or no PrEP (relative risk, 0.46; 95% confidence interval, 0.33-0.66). In 6 trials with adherence 70% or greater, the relative risk was 0.27 (95% CI, 0.19-0.39). In 7 studies on risk assessment tools for HIV infection, the instruments had moderate discrimination, though several of the studies had methodological shortcomings. As for serious adverse events, an analysis of 12 trials found no significant difference between PrEP and placebo (RR, 0.93; 95% CI, 0.77-1.12).

Dr. Chou and coauthors noted their study’s limitations, including analyzing English-language articles only and the random-effects model used to pool studies potentially returning narrow CIs. They did note, however, that the “analyses were repeated using the profile likelihood method,” which produced similar findings.

All members of the USPSTF receive travel reimbursement and an honorarium for participating in meetings. The study was funded by the Department of Health and Human Services. One of the authors reported receiving grants from the National Institutes of Health/National Institute on Drug Abuse and serving as principal investigator of NIH-funded clinical trials that received donated drugs from two pharmaceutical companies. No other conflicts of interest were reported.

SOURCE: Owens DK et al. JAMA. 2019 Jun 11. doi: 10.1001/jama.2019.6390; Chou R et al. JAMA. 2019 Jun 11. doi: 10.1001/jama.2019.2591.

Pre-exposure prophylaxis (PrEP) plus effective antiretroviral therapy should be offered to people at high risk of HIV acquisition, according to a new recommendation from the U.S. Preventive Services Task Force (USPSTF).

“The USPSTF concludes with high certainty that the net benefit of the use of PrEP to reduce the risk of acquisition of HIV infection in persons at high risk of HIV infection is substantial,” wrote first author Douglas K. Owens, MD, of Stanford (Calif.) University and fellow members of the USPSTF. The recommendation was published in JAMA.

In various at-risk groups – including men who have sex with men, people at risk through heterosexual contact, and people who inject drugs – the USPSTF recommends a Food and Drug Adminstration–approved, once-daily oral treatment with combined tenofovir disoproxil fumarate and emtricitabine. In addition, tenofovir disoproxil fumarate alone can be considered an alternative regimen for high-risk heterosexually active men and women and people who inject drugs.

This recommendation was developed after a systematic review of PrEP’s effects on HIV, adherence to the treatment, and accuracy in identifying potential treatment candidates. “The findings of this review are generally consistent with those from other recent meta-analyses that found PrEP to be effective at reducing risk of HIV infection and found greater effectiveness in trials reporting higher adherence,” wrote Roger Chou, MD, of Oregon Health & Science University in Portland and coauthors. Their study was also published in JAMA.

To comprehensively assess PrEP and thus inform the USPSTF’s HIV prevention recommendations, the researchers reviewed criteria-meeting studies on oral PrEP with tenofovir disoproxil fumarate/emtricitabine or tenofovir disoproxil fumarate monotherapy; on the diagnostic accuracy of instruments to predict HIV infection; and on PrEP adherence. The final analysis included 14 randomized clinical trials, 8 observational studies, and 7 studies of diagnostic accuracy.

In 11 of the trials, PrEP was associated with reduced risk of HIV infection versus placebo or no PrEP (relative risk, 0.46; 95% confidence interval, 0.33-0.66). In 6 trials with adherence 70% or greater, the relative risk was 0.27 (95% CI, 0.19-0.39). In 7 studies on risk assessment tools for HIV infection, the instruments had moderate discrimination, though several of the studies had methodological shortcomings. As for serious adverse events, an analysis of 12 trials found no significant difference between PrEP and placebo (RR, 0.93; 95% CI, 0.77-1.12).

Dr. Chou and coauthors noted their study’s limitations, including analyzing English-language articles only and the random-effects model used to pool studies potentially returning narrow CIs. They did note, however, that the “analyses were repeated using the profile likelihood method,” which produced similar findings.

All members of the USPSTF receive travel reimbursement and an honorarium for participating in meetings. The study was funded by the Department of Health and Human Services. One of the authors reported receiving grants from the National Institutes of Health/National Institute on Drug Abuse and serving as principal investigator of NIH-funded clinical trials that received donated drugs from two pharmaceutical companies. No other conflicts of interest were reported.

SOURCE: Owens DK et al. JAMA. 2019 Jun 11. doi: 10.1001/jama.2019.6390; Chou R et al. JAMA. 2019 Jun 11. doi: 10.1001/jama.2019.2591.

For patients with advanced cirrhosis and acute variceal bleeding, early treatment with transjugular intrahepatic portosystemic shunt (TIPS) appears to improve transplantation-free survival, according to investigators.

Early TIPS “should therefore be preferred to the current standard of care,” reported lead author Yong Lv, MD, of the Fourth Military Medical University in Xi’an, China, and colleagues, referring to standard pharmaceutical and endoscopic therapy.

“[The current standard] approach has improved patient outcomes,” the investigators wrote in the Lancet Gastroenterology & Hepatology. “However, up to 10%-20% of patients still experience treatment failure, requiring further intensive management. In such patients, [TIPS] is successful in achieving hemostasis in 90%-100% of patients. However, 6-week mortality remains high [35%-55%]. This is probably because the severity of the underlying liver disease has worsened and additional organ dysfunction may have occurred after several failed endoscopic therapy attempts.”

Previous studies have explored earlier intervention with TIPS; however, according to the investigators, these trials were inconclusive for various reasons. For example, uncovered stents and an out-of-date control therapy were employed in a trial by Monescillo et al., while a study by Garcia-Pagan et al. lacked a primary survival endpoint and has been criticized for selection bias. “Thus, whether early TIPS confers a survival benefit in a broader population remains to be assessed,” the investigators wrote.

To this end, the investigators screened 373 patients with advanced cirrhosis (Child-Pugh class B or C) and acute variceal bleeding. Of these, 132 were eligible for inclusion based on age, disease severity, willingness to participate, comorbidities, and other factors. Patients were randomized 2:1 to receive either early TIPS or standard therapy. Within 12 hours of hospital admission for the initial bleeding episode, all patients received vasoactive drugs or endoscopic band ligation and prophylactic antibiotics. Control patients continued vasoactive drugs for up to 5 days, followed by propranolol, which was titrated to reduce resting heart rate by 25% but not less than 55 beats per minute. Elective endoscopic band ligation was performed within 1-2 weeks of initial endoscopic treatment, then approximately every 2 weeks until variceal eradication, and additionally if varices reappeared. TIPS was allowed as rescue therapy. In contrast, patients in the TIPS group underwent the procedure with conscious sedation and local anesthesia within 72 hours of diagnostic endoscopy, followed by approximately 1 week of antibiotics and vasoactive drugs. TIPS revision with angioplasty or another stent placement was performed in the event of shunt dysfunction or reemergence of portal hypertensive complications. The final dataset contained 127 patients, as 3 were excluded after randomization because of exclusionary diagnoses, 1 withdrew consent, and 1 died before TIPS placement.

The primary endpoint was transplantation-free survival. The secondary endpoints were new or worsening ascites based on ultrasound score or sustained ascites necessitating paracentesis, failure to control bleeding or rebleeding, overt hepatic encephalopathy, other complications of portal hypertension, and adverse events.

After a median follow-up of 24 months, data analysis showed a survival benefit associated with early TIPS based on multiple measures. Out of 84 patients in the TIPS group, 15 (18%) died during follow-up, compared with 15 (33%) in the control group. Actuarial transplantation-free survival was also better with TIPS instead of standard therapy at 6 weeks (99% vs. 84%), 1 year (86% vs. 73%), and 2 years (79% vs. 64%). The hazard ratio for transplantation-free survival was 0.50 in favor of TIPS (P = .04). These survival advantages were maintained regardless of hepatitis B virus status or Child-Pugh/Model for End-Stage Liver Disease score.

Similarly to transplantation-free survival, patients treated with TIPS were more likely to be free of uncontrolled bleeding or rebleeding at 1 year (89% vs. 66%) and 2 years (86% vs. 57%). The associated hazard ratio for this outcome favored early TIPS (HR, 0.26; P less than .0001), and univariate and multivariate analysis confirmed an independent protective role. In further support of superiority over standard therapy, patients treated with TIPS were more likely than those in the control group to be free of new or worsening ascites at 1 year (89% vs. 57%) and 2 years (81% vs. 54%).

No significant intergroup differences were found for rates of overt hepatic encephalopathy, hepatic hydrothorax, hepatorenal syndrome, spontaneous bacterial peritonitis, hepatocellular carcinoma, serious adverse events, or nonserious adverse events. At 1 and 3 months, patients in the TIPS group had a slight increase of median bilirubin concentrations and median international normalized ratio; however, these values normalized after 6 months. A similar temporal pattern was observed in early TIPS patients with regard to median Model for End-Stage Liver Disease score.

“[The transplantation-free survival benefit of early TIPS] was probably related to better control of factors contributing to death, such as failure to control bleeding or rebleeding or new or worsening ascites, without increasing the frequency and severity of overt hepatic encephalopathy and other adverse events,” the investigators concluded. “This study provides direct evidence and greater confidence in the recommendations of current guidelines that early TIPS should be performed in high-risk patients without contraindications.

“Future studies addressing whether early TIPS can be equally recommended in Child-Pugh B and C patients are warranted,” the investigators added.

The study was funded by the National Key Technology R&D Program, Boost Program of Xijing Hospital, Optimized Overall Project of Shaanxi Province, and National Natural Science Foundation of China. The investigators reported no conflicts of interest.

SOURCE: Lv Y et al. Lancet Gastroenterol Hepatol. 2019 May 29. doi: 10.1016/S2468-1253(19)30090-1.

For patients with advanced cirrhosis and acute variceal bleeding, early treatment with transjugular intrahepatic portosystemic shunt (TIPS) appears to improve transplantation-free survival, according to investigators.

Early TIPS “should therefore be preferred to the current standard of care,” reported lead author Yong Lv, MD, of the Fourth Military Medical University in Xi’an, China, and colleagues, referring to standard pharmaceutical and endoscopic therapy.

“[The current standard] approach has improved patient outcomes,” the investigators wrote in the Lancet Gastroenterology & Hepatology. “However, up to 10%-20% of patients still experience treatment failure, requiring further intensive management. In such patients, [TIPS] is successful in achieving hemostasis in 90%-100% of patients. However, 6-week mortality remains high [35%-55%]. This is probably because the severity of the underlying liver disease has worsened and additional organ dysfunction may have occurred after several failed endoscopic therapy attempts.”

Previous studies have explored earlier intervention with TIPS; however, according to the investigators, these trials were inconclusive for various reasons. For example, uncovered stents and an out-of-date control therapy were employed in a trial by Monescillo et al., while a study by Garcia-Pagan et al. lacked a primary survival endpoint and has been criticized for selection bias. “Thus, whether early TIPS confers a survival benefit in a broader population remains to be assessed,” the investigators wrote.

To this end, the investigators screened 373 patients with advanced cirrhosis (Child-Pugh class B or C) and acute variceal bleeding. Of these, 132 were eligible for inclusion based on age, disease severity, willingness to participate, comorbidities, and other factors. Patients were randomized 2:1 to receive either early TIPS or standard therapy. Within 12 hours of hospital admission for the initial bleeding episode, all patients received vasoactive drugs or endoscopic band ligation and prophylactic antibiotics. Control patients continued vasoactive drugs for up to 5 days, followed by propranolol, which was titrated to reduce resting heart rate by 25% but not less than 55 beats per minute. Elective endoscopic band ligation was performed within 1-2 weeks of initial endoscopic treatment, then approximately every 2 weeks until variceal eradication, and additionally if varices reappeared. TIPS was allowed as rescue therapy. In contrast, patients in the TIPS group underwent the procedure with conscious sedation and local anesthesia within 72 hours of diagnostic endoscopy, followed by approximately 1 week of antibiotics and vasoactive drugs. TIPS revision with angioplasty or another stent placement was performed in the event of shunt dysfunction or reemergence of portal hypertensive complications. The final dataset contained 127 patients, as 3 were excluded after randomization because of exclusionary diagnoses, 1 withdrew consent, and 1 died before TIPS placement.

The primary endpoint was transplantation-free survival. The secondary endpoints were new or worsening ascites based on ultrasound score or sustained ascites necessitating paracentesis, failure to control bleeding or rebleeding, overt hepatic encephalopathy, other complications of portal hypertension, and adverse events.

After a median follow-up of 24 months, data analysis showed a survival benefit associated with early TIPS based on multiple measures. Out of 84 patients in the TIPS group, 15 (18%) died during follow-up, compared with 15 (33%) in the control group. Actuarial transplantation-free survival was also better with TIPS instead of standard therapy at 6 weeks (99% vs. 84%), 1 year (86% vs. 73%), and 2 years (79% vs. 64%). The hazard ratio for transplantation-free survival was 0.50 in favor of TIPS (P = .04). These survival advantages were maintained regardless of hepatitis B virus status or Child-Pugh/Model for End-Stage Liver Disease score.

Similarly to transplantation-free survival, patients treated with TIPS were more likely to be free of uncontrolled bleeding or rebleeding at 1 year (89% vs. 66%) and 2 years (86% vs. 57%). The associated hazard ratio for this outcome favored early TIPS (HR, 0.26; P less than .0001), and univariate and multivariate analysis confirmed an independent protective role. In further support of superiority over standard therapy, patients treated with TIPS were more likely than those in the control group to be free of new or worsening ascites at 1 year (89% vs. 57%) and 2 years (81% vs. 54%).

No significant intergroup differences were found for rates of overt hepatic encephalopathy, hepatic hydrothorax, hepatorenal syndrome, spontaneous bacterial peritonitis, hepatocellular carcinoma, serious adverse events, or nonserious adverse events. At 1 and 3 months, patients in the TIPS group had a slight increase of median bilirubin concentrations and median international normalized ratio; however, these values normalized after 6 months. A similar temporal pattern was observed in early TIPS patients with regard to median Model for End-Stage Liver Disease score.

“[The transplantation-free survival benefit of early TIPS] was probably related to better control of factors contributing to death, such as failure to control bleeding or rebleeding or new or worsening ascites, without increasing the frequency and severity of overt hepatic encephalopathy and other adverse events,” the investigators concluded. “This study provides direct evidence and greater confidence in the recommendations of current guidelines that early TIPS should be performed in high-risk patients without contraindications.

“Future studies addressing whether early TIPS can be equally recommended in Child-Pugh B and C patients are warranted,” the investigators added.

The study was funded by the National Key Technology R&D Program, Boost Program of Xijing Hospital, Optimized Overall Project of Shaanxi Province, and National Natural Science Foundation of China. The investigators reported no conflicts of interest.

SOURCE: Lv Y et al. Lancet Gastroenterol Hepatol. 2019 May 29. doi: 10.1016/S2468-1253(19)30090-1.

For patients with advanced cirrhosis and acute variceal bleeding, early treatment with transjugular intrahepatic portosystemic shunt (TIPS) appears to improve transplantation-free survival, according to investigators.

Early TIPS “should therefore be preferred to the current standard of care,” reported lead author Yong Lv, MD, of the Fourth Military Medical University in Xi’an, China, and colleagues, referring to standard pharmaceutical and endoscopic therapy.

“[The current standard] approach has improved patient outcomes,” the investigators wrote in the Lancet Gastroenterology & Hepatology. “However, up to 10%-20% of patients still experience treatment failure, requiring further intensive management. In such patients, [TIPS] is successful in achieving hemostasis in 90%-100% of patients. However, 6-week mortality remains high [35%-55%]. This is probably because the severity of the underlying liver disease has worsened and additional organ dysfunction may have occurred after several failed endoscopic therapy attempts.”

Previous studies have explored earlier intervention with TIPS; however, according to the investigators, these trials were inconclusive for various reasons. For example, uncovered stents and an out-of-date control therapy were employed in a trial by Monescillo et al., while a study by Garcia-Pagan et al. lacked a primary survival endpoint and has been criticized for selection bias. “Thus, whether early TIPS confers a survival benefit in a broader population remains to be assessed,” the investigators wrote.

To this end, the investigators screened 373 patients with advanced cirrhosis (Child-Pugh class B or C) and acute variceal bleeding. Of these, 132 were eligible for inclusion based on age, disease severity, willingness to participate, comorbidities, and other factors. Patients were randomized 2:1 to receive either early TIPS or standard therapy. Within 12 hours of hospital admission for the initial bleeding episode, all patients received vasoactive drugs or endoscopic band ligation and prophylactic antibiotics. Control patients continued vasoactive drugs for up to 5 days, followed by propranolol, which was titrated to reduce resting heart rate by 25% but not less than 55 beats per minute. Elective endoscopic band ligation was performed within 1-2 weeks of initial endoscopic treatment, then approximately every 2 weeks until variceal eradication, and additionally if varices reappeared. TIPS was allowed as rescue therapy. In contrast, patients in the TIPS group underwent the procedure with conscious sedation and local anesthesia within 72 hours of diagnostic endoscopy, followed by approximately 1 week of antibiotics and vasoactive drugs. TIPS revision with angioplasty or another stent placement was performed in the event of shunt dysfunction or reemergence of portal hypertensive complications. The final dataset contained 127 patients, as 3 were excluded after randomization because of exclusionary diagnoses, 1 withdrew consent, and 1 died before TIPS placement.

The primary endpoint was transplantation-free survival. The secondary endpoints were new or worsening ascites based on ultrasound score or sustained ascites necessitating paracentesis, failure to control bleeding or rebleeding, overt hepatic encephalopathy, other complications of portal hypertension, and adverse events.

After a median follow-up of 24 months, data analysis showed a survival benefit associated with early TIPS based on multiple measures. Out of 84 patients in the TIPS group, 15 (18%) died during follow-up, compared with 15 (33%) in the control group. Actuarial transplantation-free survival was also better with TIPS instead of standard therapy at 6 weeks (99% vs. 84%), 1 year (86% vs. 73%), and 2 years (79% vs. 64%). The hazard ratio for transplantation-free survival was 0.50 in favor of TIPS (P = .04). These survival advantages were maintained regardless of hepatitis B virus status or Child-Pugh/Model for End-Stage Liver Disease score.

Similarly to transplantation-free survival, patients treated with TIPS were more likely to be free of uncontrolled bleeding or rebleeding at 1 year (89% vs. 66%) and 2 years (86% vs. 57%). The associated hazard ratio for this outcome favored early TIPS (HR, 0.26; P less than .0001), and univariate and multivariate analysis confirmed an independent protective role. In further support of superiority over standard therapy, patients treated with TIPS were more likely than those in the control group to be free of new or worsening ascites at 1 year (89% vs. 57%) and 2 years (81% vs. 54%).

No significant intergroup differences were found for rates of overt hepatic encephalopathy, hepatic hydrothorax, hepatorenal syndrome, spontaneous bacterial peritonitis, hepatocellular carcinoma, serious adverse events, or nonserious adverse events. At 1 and 3 months, patients in the TIPS group had a slight increase of median bilirubin concentrations and median international normalized ratio; however, these values normalized after 6 months. A similar temporal pattern was observed in early TIPS patients with regard to median Model for End-Stage Liver Disease score.

“[The transplantation-free survival benefit of early TIPS] was probably related to better control of factors contributing to death, such as failure to control bleeding or rebleeding or new or worsening ascites, without increasing the frequency and severity of overt hepatic encephalopathy and other adverse events,” the investigators concluded. “This study provides direct evidence and greater confidence in the recommendations of current guidelines that early TIPS should be performed in high-risk patients without contraindications.

“Future studies addressing whether early TIPS can be equally recommended in Child-Pugh B and C patients are warranted,” the investigators added.

The study was funded by the National Key Technology R&D Program, Boost Program of Xijing Hospital, Optimized Overall Project of Shaanxi Province, and National Natural Science Foundation of China. The investigators reported no conflicts of interest.

SOURCE: Lv Y et al. Lancet Gastroenterol Hepatol. 2019 May 29. doi: 10.1016/S2468-1253(19)30090-1.

For current and future hospitalists, there’s no doubt that knowledge of quality improvement (QI) fundamentals is an important component of a successful practice. One physician team set out to provide their trainees with that QI foundation and described the results.

Wavebreakmedia/Thinkstock

“We believed that implementing a formal step-wise QI curriculum would not only meet the Accreditation Council of Graduate Medical Education (ACGME) requirements, but also increase residents’ knowledge of QI fundamentals and ultimately establish a culture of continuous improvement aiming to provide high-value care to our health care consumers,” said lead author J. Colt Cowdell, MD, MBA, of Mayo Clinic in Jacksonville, Fla.

Prior to any interventions, the team surveyed internal medicine residents regarding three unique patient scenarios and scored their answers. Residents were then assigned to one of five unique QI projects for the academic year in combination with a structured didactic QI curriculum.

After the structured progressive curriculum, in combination with team-based QI projects, residents were surveyed again. Results showed not only increased QI knowledge, but also improved patient safety and reduced waste.

“Keys to successful implementation included a thorough explanation of the need for this curriculum to the learners and ensuring that QI teams were multidisciplinary – residents, QI experts, nurses, techs, pharmacy, administrators, etc.,” said Dr. Cowdell.

For hospitalists in an academic setting, this work can provide a framework to incorporate QI into their residency programs. “I hope, if they have a passion for QI, they would seek out opportunities to mentor residents and help lead multidisciplinary team-based projects,” Dr. Cowdell said.
 

Reference

1. Cowdell, JC; Trautman, C; Lewis, M; Dawson, N. Integration of a Novel Quality Improvement Curriculum into an Internal Medicine Residency Program. Abstract published at Hospital Medicine 2018; April 8-11; Orlando, Fla. Abstract 54. https://www.shmabstracts.com/abstract/integration-of-a-novel-quality-improvement-curriculum-into-an-internal-medicine-residency-program/. Accessed Dec. 11, 2018.

For current and future hospitalists, there’s no doubt that knowledge of quality improvement (QI) fundamentals is an important component of a successful practice. One physician team set out to provide their trainees with that QI foundation and described the results.

Wavebreakmedia/Thinkstock

“We believed that implementing a formal step-wise QI curriculum would not only meet the Accreditation Council of Graduate Medical Education (ACGME) requirements, but also increase residents’ knowledge of QI fundamentals and ultimately establish a culture of continuous improvement aiming to provide high-value care to our health care consumers,” said lead author J. Colt Cowdell, MD, MBA, of Mayo Clinic in Jacksonville, Fla.

Prior to any interventions, the team surveyed internal medicine residents regarding three unique patient scenarios and scored their answers. Residents were then assigned to one of five unique QI projects for the academic year in combination with a structured didactic QI curriculum.

After the structured progressive curriculum, in combination with team-based QI projects, residents were surveyed again. Results showed not only increased QI knowledge, but also improved patient safety and reduced waste.

“Keys to successful implementation included a thorough explanation of the need for this curriculum to the learners and ensuring that QI teams were multidisciplinary – residents, QI experts, nurses, techs, pharmacy, administrators, etc.,” said Dr. Cowdell.

For hospitalists in an academic setting, this work can provide a framework to incorporate QI into their residency programs. “I hope, if they have a passion for QI, they would seek out opportunities to mentor residents and help lead multidisciplinary team-based projects,” Dr. Cowdell said.
 

Reference

1. Cowdell, JC; Trautman, C; Lewis, M; Dawson, N. Integration of a Novel Quality Improvement Curriculum into an Internal Medicine Residency Program. Abstract published at Hospital Medicine 2018; April 8-11; Orlando, Fla. Abstract 54. https://www.shmabstracts.com/abstract/integration-of-a-novel-quality-improvement-curriculum-into-an-internal-medicine-residency-program/. Accessed Dec. 11, 2018.

For current and future hospitalists, there’s no doubt that knowledge of quality improvement (QI) fundamentals is an important component of a successful practice. One physician team set out to provide their trainees with that QI foundation and described the results.

Wavebreakmedia/Thinkstock

“We believed that implementing a formal step-wise QI curriculum would not only meet the Accreditation Council of Graduate Medical Education (ACGME) requirements, but also increase residents’ knowledge of QI fundamentals and ultimately establish a culture of continuous improvement aiming to provide high-value care to our health care consumers,” said lead author J. Colt Cowdell, MD, MBA, of Mayo Clinic in Jacksonville, Fla.

Prior to any interventions, the team surveyed internal medicine residents regarding three unique patient scenarios and scored their answers. Residents were then assigned to one of five unique QI projects for the academic year in combination with a structured didactic QI curriculum.

After the structured progressive curriculum, in combination with team-based QI projects, residents were surveyed again. Results showed not only increased QI knowledge, but also improved patient safety and reduced waste.

“Keys to successful implementation included a thorough explanation of the need for this curriculum to the learners and ensuring that QI teams were multidisciplinary – residents, QI experts, nurses, techs, pharmacy, administrators, etc.,” said Dr. Cowdell.

For hospitalists in an academic setting, this work can provide a framework to incorporate QI into their residency programs. “I hope, if they have a passion for QI, they would seek out opportunities to mentor residents and help lead multidisciplinary team-based projects,” Dr. Cowdell said.
 

Reference

1. Cowdell, JC; Trautman, C; Lewis, M; Dawson, N. Integration of a Novel Quality Improvement Curriculum into an Internal Medicine Residency Program. Abstract published at Hospital Medicine 2018; April 8-11; Orlando, Fla. Abstract 54. https://www.shmabstracts.com/abstract/integration-of-a-novel-quality-improvement-curriculum-into-an-internal-medicine-residency-program/. Accessed Dec. 11, 2018.

Alarms from bedside continuous physiologic monitors (CPMs) occur frequently in children’s hospitals and can lead to harm. Recent studies conducted in children’s hospitals have identified alarm rates of up to 152 alarms per patient per day outside of the intensive care unit,^1-3 with as few as 1% of alarms being considered clinically important.⁴ Excessive alarms have been linked to alarm fatigue, when providers become desensitized to and may miss alarms indicating impending patient deterioration. Alarm fatigue has been identified by national patient safety organizations as a patient safety concern given the risk of patient harm.^5-7 Despite these concerns, CPMs are routinely used: up to 48% of pediatric patients in nonintensive care units at children’s hospitals are monitored.²

Although the low number of alarms that receive responses has been well-described,^8,9 the reasons why clinicians do or do not respond to alarms are unclear. A study conducted in an adult perioperative unit noted prolonged nurse response times for patients with high alarm rates.¹⁰ A second study conducted in the pediatric inpatient setting demonstrated a dose-response effect and noted progressively prolonged nurse response times with increased rates of nonactionable alarms.^4,11 Findings from another study suggested that underlying factors are highly complex and may be a result of excessive alarms, clinician characteristics, and working conditions (eg, workload and unit noise level).¹² Evidence also suggests that humans have difficulty distinguishing the importance of alarms in situations where multiple alarm tones are used, a common scenario in hospitals.^13,14 Understanding the factors that contribute to clinicians responding or not responding to CPM alarms will be crucial for addressing this serious patient safety issue.

An enhanced understanding of why nurses respond to alarms in daily practice will inform intervention development and improvement work. In the long term, this information could help improve systems for monitoring pediatric inpatients that are less prone to issues with alarm fatigue. The objective of this qualitative study, which employed structured observation, was to describe how bedside nurses think about and act upon bedside monitor alarms in a general pediatric inpatient unit.

Study Design and Setting

This prospective observational study took place on a 48-bed hospital medicine unit at a large, freestanding children’s hospital with >650 beds and >19,000 annual admissions. General Electric (Little Chalfont, United Kingdom) physiologic monitors (models Dash 3000, 4000, and 5000) were used at the time of the study, and nurses could be notified of monitor alarms in four ways: First, an in-room auditory alarm sounds. Second, a light positioned above the door outside of each patient room blinks for alarms that are at a “warning” or “critical level” (eg ventricular tachycardia or low oxygen saturation). Third, audible alarms occur at the unit’s central monitoring station. Lastly, another staff member can notify the patient’s nurse via in-person conversion or secure smart phone communication. On the study unit, CPMs are initiated and discontinued through a physician order.

This study was reviewed and approved by the hospital’s institutional review board.

Study Population

We used a purposive recruitment strategy to enroll bedside nurses working on general hospital medicine units, stratified to ensure varying levels of experience and primary shifts (eg, day vs night). We planned to conduct approximately two observations with each participating nurse and to continue collecting data until we could no longer identify new insights in terms of responses to alarms (ie, thematic saturation¹⁵). Observations were targeted to cover times of day that coincided with increased rates of distraction. These times included just prior to and after the morning and evening change of shifts (7:00 am and 7:00 pm), during morning rounds (8:00 am-12:00 pm), and heavy admission times (12:00 pm-10:00 pm). After written informed consent, a nurse was eligible for observation during his/her shift if he/she was caring for at least one monitored patient. Enrolled nurses were made aware of the general study topic but were blinded to the study team’s hypotheses.

Data Sources

Prior to data collection, the research team, which consisted of physicians, bedside nurses, research coordinators, and a human factors expert, created a system for categorizing alarm responses. Categories for observed responses were based on the location and corresponding action taken. Initial categories were developed a priori from existing literature and expanded through input from the multidisciplinary study team, then vetted with bedside staff, and finally pilot tested through >4 hours of observations, thus producing the final categories. These categories were entered into a work-sampling program (WorkStudy by Quetech Ltd., Waterloo, Ontario, Canada) to facilitate quick data recording during observations.

The hospital uses a central alarm collection software (BedMasterEx by Anandic Medical Systems, Feuerthalen, Switzerland), which permitted the collection of date, time, trigger (eg, high heart rate), and level (eg, crisis, warning) of the generated CPM alarms. Alarms collected are based on thresholds preset at the bedside monitor. The central collection software does not differentiate between accurate (eg, correctly representing the physiologic state of the patient) and inaccurate alarms.

Observation Procedure

At the time of observation, nurse demographic information (eg, primary shift worked and years working as a nurse) was obtained. A brief preobservation questionnaire was administered to collect patient information (eg, age and diagnosis) and the nurses’ perspectives on the necessity of monitors for each monitored patient in his/her care.

The observer shadowed the nurse for a two-hour block of his/her shift. During this time, nurses were instructed to “think aloud” as they responded to alarms (eg, “I notice the oxygen saturation monitor alarming off, but the probe has fallen off”). A trained observer (AML or KMT) recorded responses verbalized by the nurse and his/her reaction by selecting the appropriate category using the work-sampling software. Data were also collected on the vital sign associated with the alarm (eg, heart rate). Moreover, the observer kept written notes to provide context for electronically recorded data. Alarms that were not verbalized by the nurse were not counted. Similarly, alarms that were noted outside of the room by the nurse were not classified by vital sign unless the nurse confirmed with the bedside monitor. Observers did not adjudicate the accuracy of the alarms. The session was stopped if monitors were discontinued during the observation period. Alarm data generated by the bedside monitor were pulled for each patient room after observations were completed.

Analysis

Descriptive statistics were used to assess the percentage of each nurse response category and each alarm type (eg, heart rate and respiratory rate). The observed alarm rate was calculated by taking the total number of observed alarms (ie, alarms noted by the nurse) divided by the total number of patient-hours observed. The monitor-generated alarm rate was calculated by taking the total number of alarms from the bedside-alarm generated data divided by the number of patient-hours observed.

Electronically recorded observations using the work-sampling program were cross-referenced with hand-written field notes to assess for any discrepancies or identify relevant events not captured by the program. Three study team members (AML, KMT, and ACS) reviewed each observation independently and compared field notes to ensure accurate categorization. Discrepancies were referred to the larger study group in cases of uncertainty.

Nine nurses had monitored patients during the available observations and participated in 19 observation sessions, which included 35 monitored patients for a total of 61.3 patient-hours of observation. Nurses were observed for a median of two times each (range 1-4). The median number of monitored patients during a single observation session was two (range 1-3). Observed nurses were female with a median of eight years of experience (range 0.5-26 years). Patients represented a broad range of age categories and were hospitalized with a variety of diagnoses (Table). Nurses, when queried at the start of the observation, felt that monitors were necessary for 29 (82.9%) of the observed patients given either patient condition or unit policy.

A total of 207 observed nurse responses to alarms occurred during the study period for a rate of 3.4 responses per patient per hour. Of the total number of responses, 45 (21.7%) were noted outside of a patient room, and in 15 (33.3%) the nurse chose to go to the room. The other 162 were recorded when the nurse was present in the room when the alarm activated. Of the 177 in-person nurse responses, 50 were related to a pulse oximetry alarm, 66 were related to a heart rate alarm, and 61 were related to a respiratory rate alarm. The most common observed in-person response to an alarm involved the nurse judging that no intervention was necessary (n = 152, 73.1%). Only 14 (7% of total responses) observed in-person responses involved a clinical intervention, such as suctioning or titrating supplemental oxygen. Findings are summarized in the Figure and describe nurse-verbalized reasons to further assess (or not) and then whether the nurse chose to take action (or not) after an alarm.

We characterized responses to physiologic monitor alarms by a group of nurses with a range of experience levels. We found that most nurse responses to alarms in continuously monitored general pediatric patients involved no intervention, and further assessment was often not conducted for alarms that occurred outside of the room if the nurse noted otherwise reassuring clinical context. Observed responses occurred for 36% of alarms during the study period when compared with bedside monitor-alarm generated data. Overall, only 14 clinical interventions were noted among the observed responses. Nurses noted that they felt the monitors were necessary for 82.9% of monitored patients because of the clinical context or because of unit policy.

Our study findings highlight some potential contradictions in the current widespread use of CPMs in general pediatric units and how clinicians respond to them in practice.² First, while nurses reported that monitors were necessary for most of their patients, participating nurses deemed few alarms clinically actionable and often chose not to further assess when they noted alarms outside of the room. This is in line with findings from prior studies suggesting that clinicians overvalue the contribution of monitoring systems to patient safety.^16,17 Second, while this finding occurred in a minority of the observations, the presence of family members at the patient’s bedside was cited by nurses as a rationale for whether they responded to alarms. While family members are capable of identifying safety issues,¹⁸ formal systems to engage them in patient safety and physiologic monitoring are lacking. Finally, clinical interventions or responses to the alerts of deteriorating patients, which best represented the original intent of CPMs, were rare and accounted for just 7% of the responses. Further work elucidating why physicians and nurses choose to use CPMs may be helpful to identify interventions to reduce inappropriate monitor use and highlight gaps in frontline staff knowledge about the benefits and risks of CPM use.

Our findings provide a novel understanding of previously observed phenomena, such as long response times or nonresponses in settings with high alarm rates.^4,10 Similar to that in a prior study conducted in the pediatric setting,¹¹ alarms with an observed response constituted a minority of the total alarms that occurred in our study. This finding has previously been attributed to mental fatigue, caregiver apathy, and desensitization.⁸ However, even though a minority of observed responses in our study included an intervention, the nurse had a rationale for why the alarm did or did not need a response. This behavior and the verbalized rationale indicate that in his/her opinion, not responding to the alarm was clinically appropriate. Study participants also reflected on the difficulties of responding to alarms given the monitor system setup, in which they may not always be capable of hearing alarms for their patients. Without data from nurses regarding the alarms that had no observed response, we can only speculate; however, based on our findings, each of these factors could contribute to nonresponse. Finally, while high numbers of false alarms have been posited as an underlying cause of alarm fatigue, we noted that a majority of nonresponse was reported to be related to other clinical factors. This relationship suggests that from the nurse’s perspective, a more applicable framework for understanding alarms would be based on clinical actionability⁴ over physiologic accuracy.

In total, our findings suggest that a multifaceted approach will be necessary to improve alarm response rates. These interventions should include adjusting parameters such that alarms are highly likely to indicate a need for intervention coupled with educational interventions addressing clinician knowledge of the alarm system and bias about the actionability of alarms may improve response rates. Changes in the monitoring system setup such that nurses can easily be notified when alarms occur may also be indicated, in addition to formally engaging patients and families around response to alarms. Although secondary notification systems (eg, alarms transmitted to individual clinician’s devices) are one solution, the utilization of these systems needs to be balanced with the risks of contributing to existing alarm fatigue and the need to appropriately tailor monitoring thresholds and strategies to patients.

Our study has several limitations. First, nurses may have responded in a way they perceive to be socially desirable, and studies using in-person observers are also prone to a Hawthorne-like effect,^19-21 where the nurse may have tried to respond more frequently to alarms than usual during observations. However, given that the majority of bedside alarms did not receive a response and a substantial number of responses involved no action, these effects were likely weak. Second, we were unable to assess which alarms were accurately reflecting the patient’s physiologic status and which were not; we were also unable to link observed alarm response to monitor-recorded alarms. Third, despite the use of silent observers and an actual, rather than a simulated, clinical setting, by virtue of the data collection method we likely captured a more deliberate thought process (so-called System 2 thinking)²² rather than the subconscious processes that may predominate when nurses respond to alarms in the course of clinical care (System 1 thinking).²² Despite this limitation, our study findings, which reflect a nurse’s in-the-moment thinking, remain relevant to guiding the improvement of monitoring systems, and the development of nurse-facing interventions and education. Finally, we studied a small, purposive sample of nurses at a single hospital. Our study sample impacts the generalizability of our results and precluded a detailed analysis of the effect of nurse- and patient-level variables.

We found that nurses often deemed that no response was necessary for CPM alarms. Nurses cited contextual factors, including the duration of alarms and the presence of other providers or parents in their decision-making. Few (7%) of the alarm responses in our study included a clinical intervention. The number of observed alarm responses constituted roughly a third of the alarms recorded by bedside CPMs during the study. This result supports concerns about the nurse’s capacity to hear and process all CPM alarms given system limitations and a heavy clinical workload. Subsequent steps should include staff education, reducing overall alarm rates with appropriate monitor use and actionable alarm thresholds, and ensuring that patient alarms are easily recognizable for frontline staff.

Disclosures

The authors have no conflicts of interest to disclose.

Funding

This work was supported by the Place Outcomes Research Award from the Cincinnati Children’s Research Foundation. Dr. Brady is supported by the Agency for Healthcare Research and Quality under Award Number K08HS23827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

Alarms from bedside continuous physiologic monitors (CPMs) occur frequently in children’s hospitals and can lead to harm. Recent studies conducted in children’s hospitals have identified alarm rates of up to 152 alarms per patient per day outside of the intensive care unit,^1-3 with as few as 1% of alarms being considered clinically important.⁴ Excessive alarms have been linked to alarm fatigue, when providers become desensitized to and may miss alarms indicating impending patient deterioration. Alarm fatigue has been identified by national patient safety organizations as a patient safety concern given the risk of patient harm.^5-7 Despite these concerns, CPMs are routinely used: up to 48% of pediatric patients in nonintensive care units at children’s hospitals are monitored.²

Although the low number of alarms that receive responses has been well-described,^8,9 the reasons why clinicians do or do not respond to alarms are unclear. A study conducted in an adult perioperative unit noted prolonged nurse response times for patients with high alarm rates.¹⁰ A second study conducted in the pediatric inpatient setting demonstrated a dose-response effect and noted progressively prolonged nurse response times with increased rates of nonactionable alarms.^4,11 Findings from another study suggested that underlying factors are highly complex and may be a result of excessive alarms, clinician characteristics, and working conditions (eg, workload and unit noise level).¹² Evidence also suggests that humans have difficulty distinguishing the importance of alarms in situations where multiple alarm tones are used, a common scenario in hospitals.^13,14 Understanding the factors that contribute to clinicians responding or not responding to CPM alarms will be crucial for addressing this serious patient safety issue.

An enhanced understanding of why nurses respond to alarms in daily practice will inform intervention development and improvement work. In the long term, this information could help improve systems for monitoring pediatric inpatients that are less prone to issues with alarm fatigue. The objective of this qualitative study, which employed structured observation, was to describe how bedside nurses think about and act upon bedside monitor alarms in a general pediatric inpatient unit.

Study Design and Setting

This prospective observational study took place on a 48-bed hospital medicine unit at a large, freestanding children’s hospital with >650 beds and >19,000 annual admissions. General Electric (Little Chalfont, United Kingdom) physiologic monitors (models Dash 3000, 4000, and 5000) were used at the time of the study, and nurses could be notified of monitor alarms in four ways: First, an in-room auditory alarm sounds. Second, a light positioned above the door outside of each patient room blinks for alarms that are at a “warning” or “critical level” (eg ventricular tachycardia or low oxygen saturation). Third, audible alarms occur at the unit’s central monitoring station. Lastly, another staff member can notify the patient’s nurse via in-person conversion or secure smart phone communication. On the study unit, CPMs are initiated and discontinued through a physician order.

This study was reviewed and approved by the hospital’s institutional review board.

Study Population

We used a purposive recruitment strategy to enroll bedside nurses working on general hospital medicine units, stratified to ensure varying levels of experience and primary shifts (eg, day vs night). We planned to conduct approximately two observations with each participating nurse and to continue collecting data until we could no longer identify new insights in terms of responses to alarms (ie, thematic saturation¹⁵). Observations were targeted to cover times of day that coincided with increased rates of distraction. These times included just prior to and after the morning and evening change of shifts (7:00 am and 7:00 pm), during morning rounds (8:00 am-12:00 pm), and heavy admission times (12:00 pm-10:00 pm). After written informed consent, a nurse was eligible for observation during his/her shift if he/she was caring for at least one monitored patient. Enrolled nurses were made aware of the general study topic but were blinded to the study team’s hypotheses.

Data Sources

Prior to data collection, the research team, which consisted of physicians, bedside nurses, research coordinators, and a human factors expert, created a system for categorizing alarm responses. Categories for observed responses were based on the location and corresponding action taken. Initial categories were developed a priori from existing literature and expanded through input from the multidisciplinary study team, then vetted with bedside staff, and finally pilot tested through >4 hours of observations, thus producing the final categories. These categories were entered into a work-sampling program (WorkStudy by Quetech Ltd., Waterloo, Ontario, Canada) to facilitate quick data recording during observations.

The hospital uses a central alarm collection software (BedMasterEx by Anandic Medical Systems, Feuerthalen, Switzerland), which permitted the collection of date, time, trigger (eg, high heart rate), and level (eg, crisis, warning) of the generated CPM alarms. Alarms collected are based on thresholds preset at the bedside monitor. The central collection software does not differentiate between accurate (eg, correctly representing the physiologic state of the patient) and inaccurate alarms.

Observation Procedure

At the time of observation, nurse demographic information (eg, primary shift worked and years working as a nurse) was obtained. A brief preobservation questionnaire was administered to collect patient information (eg, age and diagnosis) and the nurses’ perspectives on the necessity of monitors for each monitored patient in his/her care.

The observer shadowed the nurse for a two-hour block of his/her shift. During this time, nurses were instructed to “think aloud” as they responded to alarms (eg, “I notice the oxygen saturation monitor alarming off, but the probe has fallen off”). A trained observer (AML or KMT) recorded responses verbalized by the nurse and his/her reaction by selecting the appropriate category using the work-sampling software. Data were also collected on the vital sign associated with the alarm (eg, heart rate). Moreover, the observer kept written notes to provide context for electronically recorded data. Alarms that were not verbalized by the nurse were not counted. Similarly, alarms that were noted outside of the room by the nurse were not classified by vital sign unless the nurse confirmed with the bedside monitor. Observers did not adjudicate the accuracy of the alarms. The session was stopped if monitors were discontinued during the observation period. Alarm data generated by the bedside monitor were pulled for each patient room after observations were completed.

Analysis

Descriptive statistics were used to assess the percentage of each nurse response category and each alarm type (eg, heart rate and respiratory rate). The observed alarm rate was calculated by taking the total number of observed alarms (ie, alarms noted by the nurse) divided by the total number of patient-hours observed. The monitor-generated alarm rate was calculated by taking the total number of alarms from the bedside-alarm generated data divided by the number of patient-hours observed.

Electronically recorded observations using the work-sampling program were cross-referenced with hand-written field notes to assess for any discrepancies or identify relevant events not captured by the program. Three study team members (AML, KMT, and ACS) reviewed each observation independently and compared field notes to ensure accurate categorization. Discrepancies were referred to the larger study group in cases of uncertainty.

Nine nurses had monitored patients during the available observations and participated in 19 observation sessions, which included 35 monitored patients for a total of 61.3 patient-hours of observation. Nurses were observed for a median of two times each (range 1-4). The median number of monitored patients during a single observation session was two (range 1-3). Observed nurses were female with a median of eight years of experience (range 0.5-26 years). Patients represented a broad range of age categories and were hospitalized with a variety of diagnoses (Table). Nurses, when queried at the start of the observation, felt that monitors were necessary for 29 (82.9%) of the observed patients given either patient condition or unit policy.

A total of 207 observed nurse responses to alarms occurred during the study period for a rate of 3.4 responses per patient per hour. Of the total number of responses, 45 (21.7%) were noted outside of a patient room, and in 15 (33.3%) the nurse chose to go to the room. The other 162 were recorded when the nurse was present in the room when the alarm activated. Of the 177 in-person nurse responses, 50 were related to a pulse oximetry alarm, 66 were related to a heart rate alarm, and 61 were related to a respiratory rate alarm. The most common observed in-person response to an alarm involved the nurse judging that no intervention was necessary (n = 152, 73.1%). Only 14 (7% of total responses) observed in-person responses involved a clinical intervention, such as suctioning or titrating supplemental oxygen. Findings are summarized in the Figure and describe nurse-verbalized reasons to further assess (or not) and then whether the nurse chose to take action (or not) after an alarm.

We characterized responses to physiologic monitor alarms by a group of nurses with a range of experience levels. We found that most nurse responses to alarms in continuously monitored general pediatric patients involved no intervention, and further assessment was often not conducted for alarms that occurred outside of the room if the nurse noted otherwise reassuring clinical context. Observed responses occurred for 36% of alarms during the study period when compared with bedside monitor-alarm generated data. Overall, only 14 clinical interventions were noted among the observed responses. Nurses noted that they felt the monitors were necessary for 82.9% of monitored patients because of the clinical context or because of unit policy.

Our study findings highlight some potential contradictions in the current widespread use of CPMs in general pediatric units and how clinicians respond to them in practice.² First, while nurses reported that monitors were necessary for most of their patients, participating nurses deemed few alarms clinically actionable and often chose not to further assess when they noted alarms outside of the room. This is in line with findings from prior studies suggesting that clinicians overvalue the contribution of monitoring systems to patient safety.^16,17 Second, while this finding occurred in a minority of the observations, the presence of family members at the patient’s bedside was cited by nurses as a rationale for whether they responded to alarms. While family members are capable of identifying safety issues,¹⁸ formal systems to engage them in patient safety and physiologic monitoring are lacking. Finally, clinical interventions or responses to the alerts of deteriorating patients, which best represented the original intent of CPMs, were rare and accounted for just 7% of the responses. Further work elucidating why physicians and nurses choose to use CPMs may be helpful to identify interventions to reduce inappropriate monitor use and highlight gaps in frontline staff knowledge about the benefits and risks of CPM use.

Our findings provide a novel understanding of previously observed phenomena, such as long response times or nonresponses in settings with high alarm rates.^4,10 Similar to that in a prior study conducted in the pediatric setting,¹¹ alarms with an observed response constituted a minority of the total alarms that occurred in our study. This finding has previously been attributed to mental fatigue, caregiver apathy, and desensitization.⁸ However, even though a minority of observed responses in our study included an intervention, the nurse had a rationale for why the alarm did or did not need a response. This behavior and the verbalized rationale indicate that in his/her opinion, not responding to the alarm was clinically appropriate. Study participants also reflected on the difficulties of responding to alarms given the monitor system setup, in which they may not always be capable of hearing alarms for their patients. Without data from nurses regarding the alarms that had no observed response, we can only speculate; however, based on our findings, each of these factors could contribute to nonresponse. Finally, while high numbers of false alarms have been posited as an underlying cause of alarm fatigue, we noted that a majority of nonresponse was reported to be related to other clinical factors. This relationship suggests that from the nurse’s perspective, a more applicable framework for understanding alarms would be based on clinical actionability⁴ over physiologic accuracy.

In total, our findings suggest that a multifaceted approach will be necessary to improve alarm response rates. These interventions should include adjusting parameters such that alarms are highly likely to indicate a need for intervention coupled with educational interventions addressing clinician knowledge of the alarm system and bias about the actionability of alarms may improve response rates. Changes in the monitoring system setup such that nurses can easily be notified when alarms occur may also be indicated, in addition to formally engaging patients and families around response to alarms. Although secondary notification systems (eg, alarms transmitted to individual clinician’s devices) are one solution, the utilization of these systems needs to be balanced with the risks of contributing to existing alarm fatigue and the need to appropriately tailor monitoring thresholds and strategies to patients.

Our study has several limitations. First, nurses may have responded in a way they perceive to be socially desirable, and studies using in-person observers are also prone to a Hawthorne-like effect,^19-21 where the nurse may have tried to respond more frequently to alarms than usual during observations. However, given that the majority of bedside alarms did not receive a response and a substantial number of responses involved no action, these effects were likely weak. Second, we were unable to assess which alarms were accurately reflecting the patient’s physiologic status and which were not; we were also unable to link observed alarm response to monitor-recorded alarms. Third, despite the use of silent observers and an actual, rather than a simulated, clinical setting, by virtue of the data collection method we likely captured a more deliberate thought process (so-called System 2 thinking)²² rather than the subconscious processes that may predominate when nurses respond to alarms in the course of clinical care (System 1 thinking).²² Despite this limitation, our study findings, which reflect a nurse’s in-the-moment thinking, remain relevant to guiding the improvement of monitoring systems, and the development of nurse-facing interventions and education. Finally, we studied a small, purposive sample of nurses at a single hospital. Our study sample impacts the generalizability of our results and precluded a detailed analysis of the effect of nurse- and patient-level variables.

We found that nurses often deemed that no response was necessary for CPM alarms. Nurses cited contextual factors, including the duration of alarms and the presence of other providers or parents in their decision-making. Few (7%) of the alarm responses in our study included a clinical intervention. The number of observed alarm responses constituted roughly a third of the alarms recorded by bedside CPMs during the study. This result supports concerns about the nurse’s capacity to hear and process all CPM alarms given system limitations and a heavy clinical workload. Subsequent steps should include staff education, reducing overall alarm rates with appropriate monitor use and actionable alarm thresholds, and ensuring that patient alarms are easily recognizable for frontline staff.

Disclosures

The authors have no conflicts of interest to disclose.

Funding

This work was supported by the Place Outcomes Research Award from the Cincinnati Children’s Research Foundation. Dr. Brady is supported by the Agency for Healthcare Research and Quality under Award Number K08HS23827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

Alarms from bedside continuous physiologic monitors (CPMs) occur frequently in children’s hospitals and can lead to harm. Recent studies conducted in children’s hospitals have identified alarm rates of up to 152 alarms per patient per day outside of the intensive care unit,^1-3 with as few as 1% of alarms being considered clinically important.⁴ Excessive alarms have been linked to alarm fatigue, when providers become desensitized to and may miss alarms indicating impending patient deterioration. Alarm fatigue has been identified by national patient safety organizations as a patient safety concern given the risk of patient harm.^5-7 Despite these concerns, CPMs are routinely used: up to 48% of pediatric patients in nonintensive care units at children’s hospitals are monitored.²

Although the low number of alarms that receive responses has been well-described,^8,9 the reasons why clinicians do or do not respond to alarms are unclear. A study conducted in an adult perioperative unit noted prolonged nurse response times for patients with high alarm rates.¹⁰ A second study conducted in the pediatric inpatient setting demonstrated a dose-response effect and noted progressively prolonged nurse response times with increased rates of nonactionable alarms.^4,11 Findings from another study suggested that underlying factors are highly complex and may be a result of excessive alarms, clinician characteristics, and working conditions (eg, workload and unit noise level).¹² Evidence also suggests that humans have difficulty distinguishing the importance of alarms in situations where multiple alarm tones are used, a common scenario in hospitals.^13,14 Understanding the factors that contribute to clinicians responding or not responding to CPM alarms will be crucial for addressing this serious patient safety issue.

An enhanced understanding of why nurses respond to alarms in daily practice will inform intervention development and improvement work. In the long term, this information could help improve systems for monitoring pediatric inpatients that are less prone to issues with alarm fatigue. The objective of this qualitative study, which employed structured observation, was to describe how bedside nurses think about and act upon bedside monitor alarms in a general pediatric inpatient unit.

Study Design and Setting

This prospective observational study took place on a 48-bed hospital medicine unit at a large, freestanding children’s hospital with >650 beds and >19,000 annual admissions. General Electric (Little Chalfont, United Kingdom) physiologic monitors (models Dash 3000, 4000, and 5000) were used at the time of the study, and nurses could be notified of monitor alarms in four ways: First, an in-room auditory alarm sounds. Second, a light positioned above the door outside of each patient room blinks for alarms that are at a “warning” or “critical level” (eg ventricular tachycardia or low oxygen saturation). Third, audible alarms occur at the unit’s central monitoring station. Lastly, another staff member can notify the patient’s nurse via in-person conversion or secure smart phone communication. On the study unit, CPMs are initiated and discontinued through a physician order.

This study was reviewed and approved by the hospital’s institutional review board.

Study Population

We used a purposive recruitment strategy to enroll bedside nurses working on general hospital medicine units, stratified to ensure varying levels of experience and primary shifts (eg, day vs night). We planned to conduct approximately two observations with each participating nurse and to continue collecting data until we could no longer identify new insights in terms of responses to alarms (ie, thematic saturation¹⁵). Observations were targeted to cover times of day that coincided with increased rates of distraction. These times included just prior to and after the morning and evening change of shifts (7:00 am and 7:00 pm), during morning rounds (8:00 am-12:00 pm), and heavy admission times (12:00 pm-10:00 pm). After written informed consent, a nurse was eligible for observation during his/her shift if he/she was caring for at least one monitored patient. Enrolled nurses were made aware of the general study topic but were blinded to the study team’s hypotheses.

Data Sources

Prior to data collection, the research team, which consisted of physicians, bedside nurses, research coordinators, and a human factors expert, created a system for categorizing alarm responses. Categories for observed responses were based on the location and corresponding action taken. Initial categories were developed a priori from existing literature and expanded through input from the multidisciplinary study team, then vetted with bedside staff, and finally pilot tested through >4 hours of observations, thus producing the final categories. These categories were entered into a work-sampling program (WorkStudy by Quetech Ltd., Waterloo, Ontario, Canada) to facilitate quick data recording during observations.

The hospital uses a central alarm collection software (BedMasterEx by Anandic Medical Systems, Feuerthalen, Switzerland), which permitted the collection of date, time, trigger (eg, high heart rate), and level (eg, crisis, warning) of the generated CPM alarms. Alarms collected are based on thresholds preset at the bedside monitor. The central collection software does not differentiate between accurate (eg, correctly representing the physiologic state of the patient) and inaccurate alarms.

Observation Procedure

At the time of observation, nurse demographic information (eg, primary shift worked and years working as a nurse) was obtained. A brief preobservation questionnaire was administered to collect patient information (eg, age and diagnosis) and the nurses’ perspectives on the necessity of monitors for each monitored patient in his/her care.

The observer shadowed the nurse for a two-hour block of his/her shift. During this time, nurses were instructed to “think aloud” as they responded to alarms (eg, “I notice the oxygen saturation monitor alarming off, but the probe has fallen off”). A trained observer (AML or KMT) recorded responses verbalized by the nurse and his/her reaction by selecting the appropriate category using the work-sampling software. Data were also collected on the vital sign associated with the alarm (eg, heart rate). Moreover, the observer kept written notes to provide context for electronically recorded data. Alarms that were not verbalized by the nurse were not counted. Similarly, alarms that were noted outside of the room by the nurse were not classified by vital sign unless the nurse confirmed with the bedside monitor. Observers did not adjudicate the accuracy of the alarms. The session was stopped if monitors were discontinued during the observation period. Alarm data generated by the bedside monitor were pulled for each patient room after observations were completed.

Analysis

Descriptive statistics were used to assess the percentage of each nurse response category and each alarm type (eg, heart rate and respiratory rate). The observed alarm rate was calculated by taking the total number of observed alarms (ie, alarms noted by the nurse) divided by the total number of patient-hours observed. The monitor-generated alarm rate was calculated by taking the total number of alarms from the bedside-alarm generated data divided by the number of patient-hours observed.

Electronically recorded observations using the work-sampling program were cross-referenced with hand-written field notes to assess for any discrepancies or identify relevant events not captured by the program. Three study team members (AML, KMT, and ACS) reviewed each observation independently and compared field notes to ensure accurate categorization. Discrepancies were referred to the larger study group in cases of uncertainty.

Nine nurses had monitored patients during the available observations and participated in 19 observation sessions, which included 35 monitored patients for a total of 61.3 patient-hours of observation. Nurses were observed for a median of two times each (range 1-4). The median number of monitored patients during a single observation session was two (range 1-3). Observed nurses were female with a median of eight years of experience (range 0.5-26 years). Patients represented a broad range of age categories and were hospitalized with a variety of diagnoses (Table). Nurses, when queried at the start of the observation, felt that monitors were necessary for 29 (82.9%) of the observed patients given either patient condition or unit policy.

A total of 207 observed nurse responses to alarms occurred during the study period for a rate of 3.4 responses per patient per hour. Of the total number of responses, 45 (21.7%) were noted outside of a patient room, and in 15 (33.3%) the nurse chose to go to the room. The other 162 were recorded when the nurse was present in the room when the alarm activated. Of the 177 in-person nurse responses, 50 were related to a pulse oximetry alarm, 66 were related to a heart rate alarm, and 61 were related to a respiratory rate alarm. The most common observed in-person response to an alarm involved the nurse judging that no intervention was necessary (n = 152, 73.1%). Only 14 (7% of total responses) observed in-person responses involved a clinical intervention, such as suctioning or titrating supplemental oxygen. Findings are summarized in the Figure and describe nurse-verbalized reasons to further assess (or not) and then whether the nurse chose to take action (or not) after an alarm.

We characterized responses to physiologic monitor alarms by a group of nurses with a range of experience levels. We found that most nurse responses to alarms in continuously monitored general pediatric patients involved no intervention, and further assessment was often not conducted for alarms that occurred outside of the room if the nurse noted otherwise reassuring clinical context. Observed responses occurred for 36% of alarms during the study period when compared with bedside monitor-alarm generated data. Overall, only 14 clinical interventions were noted among the observed responses. Nurses noted that they felt the monitors were necessary for 82.9% of monitored patients because of the clinical context or because of unit policy.

Our study findings highlight some potential contradictions in the current widespread use of CPMs in general pediatric units and how clinicians respond to them in practice.² First, while nurses reported that monitors were necessary for most of their patients, participating nurses deemed few alarms clinically actionable and often chose not to further assess when they noted alarms outside of the room. This is in line with findings from prior studies suggesting that clinicians overvalue the contribution of monitoring systems to patient safety.^16,17 Second, while this finding occurred in a minority of the observations, the presence of family members at the patient’s bedside was cited by nurses as a rationale for whether they responded to alarms. While family members are capable of identifying safety issues,¹⁸ formal systems to engage them in patient safety and physiologic monitoring are lacking. Finally, clinical interventions or responses to the alerts of deteriorating patients, which best represented the original intent of CPMs, were rare and accounted for just 7% of the responses. Further work elucidating why physicians and nurses choose to use CPMs may be helpful to identify interventions to reduce inappropriate monitor use and highlight gaps in frontline staff knowledge about the benefits and risks of CPM use.

Our findings provide a novel understanding of previously observed phenomena, such as long response times or nonresponses in settings with high alarm rates.^4,10 Similar to that in a prior study conducted in the pediatric setting,¹¹ alarms with an observed response constituted a minority of the total alarms that occurred in our study. This finding has previously been attributed to mental fatigue, caregiver apathy, and desensitization.⁸ However, even though a minority of observed responses in our study included an intervention, the nurse had a rationale for why the alarm did or did not need a response. This behavior and the verbalized rationale indicate that in his/her opinion, not responding to the alarm was clinically appropriate. Study participants also reflected on the difficulties of responding to alarms given the monitor system setup, in which they may not always be capable of hearing alarms for their patients. Without data from nurses regarding the alarms that had no observed response, we can only speculate; however, based on our findings, each of these factors could contribute to nonresponse. Finally, while high numbers of false alarms have been posited as an underlying cause of alarm fatigue, we noted that a majority of nonresponse was reported to be related to other clinical factors. This relationship suggests that from the nurse’s perspective, a more applicable framework for understanding alarms would be based on clinical actionability⁴ over physiologic accuracy.

In total, our findings suggest that a multifaceted approach will be necessary to improve alarm response rates. These interventions should include adjusting parameters such that alarms are highly likely to indicate a need for intervention coupled with educational interventions addressing clinician knowledge of the alarm system and bias about the actionability of alarms may improve response rates. Changes in the monitoring system setup such that nurses can easily be notified when alarms occur may also be indicated, in addition to formally engaging patients and families around response to alarms. Although secondary notification systems (eg, alarms transmitted to individual clinician’s devices) are one solution, the utilization of these systems needs to be balanced with the risks of contributing to existing alarm fatigue and the need to appropriately tailor monitoring thresholds and strategies to patients.

Our study has several limitations. First, nurses may have responded in a way they perceive to be socially desirable, and studies using in-person observers are also prone to a Hawthorne-like effect,^19-21 where the nurse may have tried to respond more frequently to alarms than usual during observations. However, given that the majority of bedside alarms did not receive a response and a substantial number of responses involved no action, these effects were likely weak. Second, we were unable to assess which alarms were accurately reflecting the patient’s physiologic status and which were not; we were also unable to link observed alarm response to monitor-recorded alarms. Third, despite the use of silent observers and an actual, rather than a simulated, clinical setting, by virtue of the data collection method we likely captured a more deliberate thought process (so-called System 2 thinking)²² rather than the subconscious processes that may predominate when nurses respond to alarms in the course of clinical care (System 1 thinking).²² Despite this limitation, our study findings, which reflect a nurse’s in-the-moment thinking, remain relevant to guiding the improvement of monitoring systems, and the development of nurse-facing interventions and education. Finally, we studied a small, purposive sample of nurses at a single hospital. Our study sample impacts the generalizability of our results and precluded a detailed analysis of the effect of nurse- and patient-level variables.

We found that nurses often deemed that no response was necessary for CPM alarms. Nurses cited contextual factors, including the duration of alarms and the presence of other providers or parents in their decision-making. Few (7%) of the alarm responses in our study included a clinical intervention. The number of observed alarm responses constituted roughly a third of the alarms recorded by bedside CPMs during the study. This result supports concerns about the nurse’s capacity to hear and process all CPM alarms given system limitations and a heavy clinical workload. Subsequent steps should include staff education, reducing overall alarm rates with appropriate monitor use and actionable alarm thresholds, and ensuring that patient alarms are easily recognizable for frontline staff.

Disclosures

The authors have no conflicts of interest to disclose.

Funding

This work was supported by the Place Outcomes Research Award from the Cincinnati Children’s Research Foundation. Dr. Brady is supported by the Agency for Healthcare Research and Quality under Award Number K08HS23827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

Automated early warning systems (EWSs) use data inputs to recognize clinical states requiring time-sensitive intervention and then generate notifications through different modalities to clinicians. EWSs serve as common tools for improving the recognition and treatment of important clinical states such as sepsis. However, despite the early enthusiasm, these warning systems have often yielded disappointing outcomes. In sepsis, for example, EWSs have shown mixed results in clinical trials, and concerns regarding the overuse of EWSs in diagnosing sepsis have grown.^1-4 We argue that inattention to the importance of timing in EWS training and evaluation provides one reason that EWSs have underperformed. Thus, to improve care, a warning system must not only identify the clinical state accurately, but it must also do so in a sufficiently timely manner to implement the associated interventions, such as administration of antibiotics for sepsis. Although the literature has occasionally highlighted the importance of timing in electronic surveillance systems, no one has linked the temporal dependence of performance metrics and intervention feasibility to the failure of such warning systems and explained how to operationalize timing in their development.^5-8 Using sepsis as an example, we explain why timing is important and propose new metrics and strategies for training and evaluating EWS models. EWSs are divided into two types: detection systems that recognize critical illnesses at a particular moment and prediction systems that estimate risk of deterioration over varying time frames.⁹ We focus primarily on detection systems, but our analysis is also important for prediction systems, which we will discuss in the last section.

EWS metrics have evolved from focusing on crude measures of discrimination to more clinically relevant metrics, such as the positive predictive value (PPV). The common performance metrics, including the c-statistic, evaluate the performance of EWSs in distinguishing events from nonevents, such as the presence or absence of sepsis in hospitalized patients. However, the c-statistic does not account for disease prevalence. A given c-statistic is compatible with a wide range of PPVs; a low PPV may limit an EWS’s usefulness to promote interventions and generate increased alert fatigue.¹⁰

However, the PPV, although important, provides no information on the timing of state recognition in relation to clinical time zero. Time zero is the first moment at which a critical state can be recognized based on available data and current medical science. Different approaches, including laboratory values, clinical assessments, retrospective chart reviews, triage times, and others, have been used to measure time zero.^8,11-13 All these approaches feature advantages and disadvantages; the evaluation of timing will exhibit sensitivity to the approach used.¹⁴ Further work is needed to gain additional insights into the measurement of time zero.

a) 50% true cases of sepsis, with a mean time of 35 minutes after clinical time zero;

b) 50% true cases, with a mean time of 60 minutes before clinical time zero (prediction EWS);

c) 50% true cases of sepsis, with a mean time of 1.3 days since clinical time zero, but with 70% of these cases undiagnosed at the time of EWS detection;

d) 50% true cases of cases, with mean time of 1.3 days since clinical time zero, that is, all cases among those promptly detected and treated through routine clinician oversight.

Each of these situations features differing clinical utility to help meet the hospital objective of increasing early administration of antibiotics. More generally, three dimensions of timing are important for detection systems. The first dimension is the timing of detection relative to time zero. The second is the timing relative to ”real-world” clinician detection. The third is timing with respect to the associated clinical objective. For a given PPV, an EWS performs better when detecting a state (1) at, near, or in advance of time zero, (2) prior to clinician detection, and (3) sufficiently in advance of an operational objective to promote change. On the other hand, when an EWS consistently sends alerts after clinician action, it serves a lesser purpose and risks causing alert fatigue; such cases have been described in studies.¹⁵

Acknowledging the importance of timing features implications for researchers and health system leaders. Researchers who develop EWS should include how these systems perform relative to both time zero and critical milestones in the clinical course. Operational leadership should understand the trade-offs that occur between alert fatigue (through lower PPV at the margin with earlier detection) and lead time to implement an intervention. Navigating these trade-offs involves a complex organizational decision. The “number needed to evaluate” is one way to quantify this fatigue factor.¹⁶ Such a measure gives a sense of the number of cases a clinician will need to evaluate per event. Collaborations between clinical leadership, operational leadership, and data scientists are needed to determine how to evaluate individual systems.

A good metric should capture the three important dimensions of timing while retaining intuitiveness to clinicians and leadership. One graphical option involves plotting the PPVs over time and relative to the clinical state evolution (Figure). This PPV-over-time curve shows when true positives occur relative to the time course of sepsis, including the three major dimensions of timing. This curve can also show a “clinically important window (CIW)”, which is bounded on the right by the latest point in time when recognition could still meet the clinical objective. For sepsis, the curve might be bounded at 2.5 hours to meet an objective of antibiotics within three hours, with the assumption that 0.5 hour is needed for a response. For detection systems, the window would be bounded on the left by clinical time zero. The graph can also designate the point when most cases of sepsis have been recognized clinically with historical data. The Figure depicts an example curve for a detection model.

The metrics derived from this curve may be used alongside the PPV for training and evaluation. Often, adjusting the PPV for its relationship to time zero and the CIW will aid in recognizing the existence of a time beyond which detection fails to help achieve the intended intervention. Detection beyond the window should not credited as a true positive if it fails to facilitate the objective. One option is to credit detection at or before time zero as one and discount later detection by the delay from time zero. More specifically, a true positive could be discounted by the difference between the end of the CIW and the moment of detection divided by the CIW length. This discounted PPV could be displayed alongside the PPV to gauge the temporal dimension of performance and be used for training.

The use of timing places additional demands on validation owing to the need for a time-based gold standard. In such a case, the unit of analysis in system development might not be the patient encounter but rather the patient-hour or patient-15-minute epoch, depending on how frequently the EWS updates risk information and may alert. By contrast, the sepsis detection models used in administrative databases rely on an encounter-level PPV, which provides more limited information compared with real-time EWSs.¹⁷ When time zero cannot be measured, alternatives may be used to capture several dimensions of timing; these alternatives include measurement of the percentage of cases that recognize the event prior to clinicians.¹⁵

Detection systems face the limitation that they lack the capability to identify a state before its occurrence. Prediction systems are more likely to be actionable, as they provide more lead time for intervention, but accurate prediction models are also more difficult to develop. With a predictive system, an additional dimension of timing becomes important: the time horizon for prediction. Prediction models may be trained to recognize a state within a specific time frame (eg, 6, 12, or 24 hours), and test characteristics, including PPV, may vary with the window.¹⁸ A given PPV (of eventual development of sepsis) is compatible with varying time windows and thus again lacks important information on performance.

The timing relative to clinical time zero remains important for prediction. For a predictive EWS, the graph in the figure may be expected to shift to the left. Models with good performance will occasionally send an alert after time zero. For a prediction system with a time horizon of six hours, it is more useful to have alerts occur a mean time of four hours prior to time zero than four minutes prior.

Improving the clinical utility of EWSs requires better measurement of timing. Researchers should incorporate timing into system development, and operational leaders should be cognizant of timing during implementation. Specific steps should include devising better strategies to estimate the relationship of state recognition to clinical time zero and developing methods to discount recognition when it occurs too late to be actionable.

Disclosures

Dr. Rolnick is a consultant to Tuple Health, Inc. and was previously a part-time employee of Acumen, LLC. Dr. Weissman has nothing to disclose.

Automated early warning systems (EWSs) use data inputs to recognize clinical states requiring time-sensitive intervention and then generate notifications through different modalities to clinicians. EWSs serve as common tools for improving the recognition and treatment of important clinical states such as sepsis. However, despite the early enthusiasm, these warning systems have often yielded disappointing outcomes. In sepsis, for example, EWSs have shown mixed results in clinical trials, and concerns regarding the overuse of EWSs in diagnosing sepsis have grown.^1-4 We argue that inattention to the importance of timing in EWS training and evaluation provides one reason that EWSs have underperformed. Thus, to improve care, a warning system must not only identify the clinical state accurately, but it must also do so in a sufficiently timely manner to implement the associated interventions, such as administration of antibiotics for sepsis. Although the literature has occasionally highlighted the importance of timing in electronic surveillance systems, no one has linked the temporal dependence of performance metrics and intervention feasibility to the failure of such warning systems and explained how to operationalize timing in their development.^5-8 Using sepsis as an example, we explain why timing is important and propose new metrics and strategies for training and evaluating EWS models. EWSs are divided into two types: detection systems that recognize critical illnesses at a particular moment and prediction systems that estimate risk of deterioration over varying time frames.⁹ We focus primarily on detection systems, but our analysis is also important for prediction systems, which we will discuss in the last section.

EWS metrics have evolved from focusing on crude measures of discrimination to more clinically relevant metrics, such as the positive predictive value (PPV). The common performance metrics, including the c-statistic, evaluate the performance of EWSs in distinguishing events from nonevents, such as the presence or absence of sepsis in hospitalized patients. However, the c-statistic does not account for disease prevalence. A given c-statistic is compatible with a wide range of PPVs; a low PPV may limit an EWS’s usefulness to promote interventions and generate increased alert fatigue.¹⁰

However, the PPV, although important, provides no information on the timing of state recognition in relation to clinical time zero. Time zero is the first moment at which a critical state can be recognized based on available data and current medical science. Different approaches, including laboratory values, clinical assessments, retrospective chart reviews, triage times, and others, have been used to measure time zero.^8,11-13 All these approaches feature advantages and disadvantages; the evaluation of timing will exhibit sensitivity to the approach used.¹⁴ Further work is needed to gain additional insights into the measurement of time zero.

a) 50% true cases of sepsis, with a mean time of 35 minutes after clinical time zero;

b) 50% true cases, with a mean time of 60 minutes before clinical time zero (prediction EWS);

c) 50% true cases of sepsis, with a mean time of 1.3 days since clinical time zero, but with 70% of these cases undiagnosed at the time of EWS detection;

d) 50% true cases of cases, with mean time of 1.3 days since clinical time zero, that is, all cases among those promptly detected and treated through routine clinician oversight.

Each of these situations features differing clinical utility to help meet the hospital objective of increasing early administration of antibiotics. More generally, three dimensions of timing are important for detection systems. The first dimension is the timing of detection relative to time zero. The second is the timing relative to ”real-world” clinician detection. The third is timing with respect to the associated clinical objective. For a given PPV, an EWS performs better when detecting a state (1) at, near, or in advance of time zero, (2) prior to clinician detection, and (3) sufficiently in advance of an operational objective to promote change. On the other hand, when an EWS consistently sends alerts after clinician action, it serves a lesser purpose and risks causing alert fatigue; such cases have been described in studies.¹⁵

Acknowledging the importance of timing features implications for researchers and health system leaders. Researchers who develop EWS should include how these systems perform relative to both time zero and critical milestones in the clinical course. Operational leadership should understand the trade-offs that occur between alert fatigue (through lower PPV at the margin with earlier detection) and lead time to implement an intervention. Navigating these trade-offs involves a complex organizational decision. The “number needed to evaluate” is one way to quantify this fatigue factor.¹⁶ Such a measure gives a sense of the number of cases a clinician will need to evaluate per event. Collaborations between clinical leadership, operational leadership, and data scientists are needed to determine how to evaluate individual systems.

A good metric should capture the three important dimensions of timing while retaining intuitiveness to clinicians and leadership. One graphical option involves plotting the PPVs over time and relative to the clinical state evolution (Figure). This PPV-over-time curve shows when true positives occur relative to the time course of sepsis, including the three major dimensions of timing. This curve can also show a “clinically important window (CIW)”, which is bounded on the right by the latest point in time when recognition could still meet the clinical objective. For sepsis, the curve might be bounded at 2.5 hours to meet an objective of antibiotics within three hours, with the assumption that 0.5 hour is needed for a response. For detection systems, the window would be bounded on the left by clinical time zero. The graph can also designate the point when most cases of sepsis have been recognized clinically with historical data. The Figure depicts an example curve for a detection model.

The metrics derived from this curve may be used alongside the PPV for training and evaluation. Often, adjusting the PPV for its relationship to time zero and the CIW will aid in recognizing the existence of a time beyond which detection fails to help achieve the intended intervention. Detection beyond the window should not credited as a true positive if it fails to facilitate the objective. One option is to credit detection at or before time zero as one and discount later detection by the delay from time zero. More specifically, a true positive could be discounted by the difference between the end of the CIW and the moment of detection divided by the CIW length. This discounted PPV could be displayed alongside the PPV to gauge the temporal dimension of performance and be used for training.

The use of timing places additional demands on validation owing to the need for a time-based gold standard. In such a case, the unit of analysis in system development might not be the patient encounter but rather the patient-hour or patient-15-minute epoch, depending on how frequently the EWS updates risk information and may alert. By contrast, the sepsis detection models used in administrative databases rely on an encounter-level PPV, which provides more limited information compared with real-time EWSs.¹⁷ When time zero cannot be measured, alternatives may be used to capture several dimensions of timing; these alternatives include measurement of the percentage of cases that recognize the event prior to clinicians.¹⁵

Detection systems face the limitation that they lack the capability to identify a state before its occurrence. Prediction systems are more likely to be actionable, as they provide more lead time for intervention, but accurate prediction models are also more difficult to develop. With a predictive system, an additional dimension of timing becomes important: the time horizon for prediction. Prediction models may be trained to recognize a state within a specific time frame (eg, 6, 12, or 24 hours), and test characteristics, including PPV, may vary with the window.¹⁸ A given PPV (of eventual development of sepsis) is compatible with varying time windows and thus again lacks important information on performance.

The timing relative to clinical time zero remains important for prediction. For a predictive EWS, the graph in the figure may be expected to shift to the left. Models with good performance will occasionally send an alert after time zero. For a prediction system with a time horizon of six hours, it is more useful to have alerts occur a mean time of four hours prior to time zero than four minutes prior.

Improving the clinical utility of EWSs requires better measurement of timing. Researchers should incorporate timing into system development, and operational leaders should be cognizant of timing during implementation. Specific steps should include devising better strategies to estimate the relationship of state recognition to clinical time zero and developing methods to discount recognition when it occurs too late to be actionable.

Disclosures

Dr. Rolnick is a consultant to Tuple Health, Inc. and was previously a part-time employee of Acumen, LLC. Dr. Weissman has nothing to disclose.

Automated early warning systems (EWSs) use data inputs to recognize clinical states requiring time-sensitive intervention and then generate notifications through different modalities to clinicians. EWSs serve as common tools for improving the recognition and treatment of important clinical states such as sepsis. However, despite the early enthusiasm, these warning systems have often yielded disappointing outcomes. In sepsis, for example, EWSs have shown mixed results in clinical trials, and concerns regarding the overuse of EWSs in diagnosing sepsis have grown.^1-4 We argue that inattention to the importance of timing in EWS training and evaluation provides one reason that EWSs have underperformed. Thus, to improve care, a warning system must not only identify the clinical state accurately, but it must also do so in a sufficiently timely manner to implement the associated interventions, such as administration of antibiotics for sepsis. Although the literature has occasionally highlighted the importance of timing in electronic surveillance systems, no one has linked the temporal dependence of performance metrics and intervention feasibility to the failure of such warning systems and explained how to operationalize timing in their development.^5-8 Using sepsis as an example, we explain why timing is important and propose new metrics and strategies for training and evaluating EWS models. EWSs are divided into two types: detection systems that recognize critical illnesses at a particular moment and prediction systems that estimate risk of deterioration over varying time frames.⁹ We focus primarily on detection systems, but our analysis is also important for prediction systems, which we will discuss in the last section.

EWS metrics have evolved from focusing on crude measures of discrimination to more clinically relevant metrics, such as the positive predictive value (PPV). The common performance metrics, including the c-statistic, evaluate the performance of EWSs in distinguishing events from nonevents, such as the presence or absence of sepsis in hospitalized patients. However, the c-statistic does not account for disease prevalence. A given c-statistic is compatible with a wide range of PPVs; a low PPV may limit an EWS’s usefulness to promote interventions and generate increased alert fatigue.¹⁰

However, the PPV, although important, provides no information on the timing of state recognition in relation to clinical time zero. Time zero is the first moment at which a critical state can be recognized based on available data and current medical science. Different approaches, including laboratory values, clinical assessments, retrospective chart reviews, triage times, and others, have been used to measure time zero.^8,11-13 All these approaches feature advantages and disadvantages; the evaluation of timing will exhibit sensitivity to the approach used.¹⁴ Further work is needed to gain additional insights into the measurement of time zero.

a) 50% true cases of sepsis, with a mean time of 35 minutes after clinical time zero;

b) 50% true cases, with a mean time of 60 minutes before clinical time zero (prediction EWS);

c) 50% true cases of sepsis, with a mean time of 1.3 days since clinical time zero, but with 70% of these cases undiagnosed at the time of EWS detection;

d) 50% true cases of cases, with mean time of 1.3 days since clinical time zero, that is, all cases among those promptly detected and treated through routine clinician oversight.

Each of these situations features differing clinical utility to help meet the hospital objective of increasing early administration of antibiotics. More generally, three dimensions of timing are important for detection systems. The first dimension is the timing of detection relative to time zero. The second is the timing relative to ”real-world” clinician detection. The third is timing with respect to the associated clinical objective. For a given PPV, an EWS performs better when detecting a state (1) at, near, or in advance of time zero, (2) prior to clinician detection, and (3) sufficiently in advance of an operational objective to promote change. On the other hand, when an EWS consistently sends alerts after clinician action, it serves a lesser purpose and risks causing alert fatigue; such cases have been described in studies.¹⁵

Acknowledging the importance of timing features implications for researchers and health system leaders. Researchers who develop EWS should include how these systems perform relative to both time zero and critical milestones in the clinical course. Operational leadership should understand the trade-offs that occur between alert fatigue (through lower PPV at the margin with earlier detection) and lead time to implement an intervention. Navigating these trade-offs involves a complex organizational decision. The “number needed to evaluate” is one way to quantify this fatigue factor.¹⁶ Such a measure gives a sense of the number of cases a clinician will need to evaluate per event. Collaborations between clinical leadership, operational leadership, and data scientists are needed to determine how to evaluate individual systems.

A good metric should capture the three important dimensions of timing while retaining intuitiveness to clinicians and leadership. One graphical option involves plotting the PPVs over time and relative to the clinical state evolution (Figure). This PPV-over-time curve shows when true positives occur relative to the time course of sepsis, including the three major dimensions of timing. This curve can also show a “clinically important window (CIW)”, which is bounded on the right by the latest point in time when recognition could still meet the clinical objective. For sepsis, the curve might be bounded at 2.5 hours to meet an objective of antibiotics within three hours, with the assumption that 0.5 hour is needed for a response. For detection systems, the window would be bounded on the left by clinical time zero. The graph can also designate the point when most cases of sepsis have been recognized clinically with historical data. The Figure depicts an example curve for a detection model.

The metrics derived from this curve may be used alongside the PPV for training and evaluation. Often, adjusting the PPV for its relationship to time zero and the CIW will aid in recognizing the existence of a time beyond which detection fails to help achieve the intended intervention. Detection beyond the window should not credited as a true positive if it fails to facilitate the objective. One option is to credit detection at or before time zero as one and discount later detection by the delay from time zero. More specifically, a true positive could be discounted by the difference between the end of the CIW and the moment of detection divided by the CIW length. This discounted PPV could be displayed alongside the PPV to gauge the temporal dimension of performance and be used for training.

The use of timing places additional demands on validation owing to the need for a time-based gold standard. In such a case, the unit of analysis in system development might not be the patient encounter but rather the patient-hour or patient-15-minute epoch, depending on how frequently the EWS updates risk information and may alert. By contrast, the sepsis detection models used in administrative databases rely on an encounter-level PPV, which provides more limited information compared with real-time EWSs.¹⁷ When time zero cannot be measured, alternatives may be used to capture several dimensions of timing; these alternatives include measurement of the percentage of cases that recognize the event prior to clinicians.¹⁵

Detection systems face the limitation that they lack the capability to identify a state before its occurrence. Prediction systems are more likely to be actionable, as they provide more lead time for intervention, but accurate prediction models are also more difficult to develop. With a predictive system, an additional dimension of timing becomes important: the time horizon for prediction. Prediction models may be trained to recognize a state within a specific time frame (eg, 6, 12, or 24 hours), and test characteristics, including PPV, may vary with the window.¹⁸ A given PPV (of eventual development of sepsis) is compatible with varying time windows and thus again lacks important information on performance.

The timing relative to clinical time zero remains important for prediction. For a predictive EWS, the graph in the figure may be expected to shift to the left. Models with good performance will occasionally send an alert after time zero. For a prediction system with a time horizon of six hours, it is more useful to have alerts occur a mean time of four hours prior to time zero than four minutes prior.

Improving the clinical utility of EWSs requires better measurement of timing. Researchers should incorporate timing into system development, and operational leaders should be cognizant of timing during implementation. Specific steps should include devising better strategies to estimate the relationship of state recognition to clinical time zero and developing methods to discount recognition when it occurs too late to be actionable.

Disclosures

Dr. Rolnick is a consultant to Tuple Health, Inc. and was previously a part-time employee of Acumen, LLC. Dr. Weissman has nothing to disclose.

A 58-year-old man with a remote history of endocarditis and no prior injection drug use was admitted to the inpatient medicine service with fever and concern for recurrent endocarditis. A transthoracic echocardiogram was unremarkable and the patient remained clinically stable. A transesophageal echocardiogram (TEE) was scheduled for the following morning, but during nursing rounds, the patient was missing from his room. Multiple staff members searched for the patient and eventually located him in the hospital lobby drinking a cup of coffee purchased from the cafeteria. Despite his opposition, he was escorted back to his room and advised to not leave the floor again. Later that day, the patient became frustrated and left the hospital before his scheduled TEE. He was subsequently lost to follow-up.

Patients are admitted to the hospital based upon a medical determination that the patient requires acute observation, evaluation, or treatment. Once admitted, healthcare providers may impose restrictions on the patient’s movement in the hospital, such as restrictions on leaving their assigned floor. Managing the movement of hospitalized patients poses significant challenges for the clinical staff because of the difficulty of providing a treatment environment that ensures safe and efficient delivery of care while promoting patients’ preferences for an unrestrictive environment that respects their independence.^1,2 Broad limits may make it easier for staff to care for patients and reduce concerns about liability, but they may also frustrate patients who may be medically, psychiatrically, and physically stable and do not require stringent monitoring (eg, completing a course of intravenous antibiotics or awaiting placement at outside facilities).

Although this issue has broad implications for patient safety and hospital liability, authoritative guidance and evidence-based literature are lacking. Without clear guidelines, healthcare staff members are likely to spend more time in managing each individual request to leave the floor because they do not have a systematic strategy for making fair and consistent decisions. Here, we describe the patient and institutional considerations when managing patient movement in the hospital. We refer to “patient movement” specifically as a patient’s choice to move to different locations within the hospital, but outside of their assigned room and/or floor. This does not include scheduled, supervised ambulation activities, such as physical therapy.

Practices that promote patient movement offer significant benefits and risks. Enhancing movement is likely to reduce the “physiologic disruption”³ of hospitalization while improving patients’ overall satisfaction and alignment with patient-centered care. Liberalized movement also promotes independence and ambulation that reduces the rate of physical deconditioning.⁴

Despite theoretical benefits, hospitals may be more concerned about adverse events related to patient movement, such as falls, the use of illicit substances, or elopement. Given that hospitals may be legally⁵ and financially responsible⁶ for adverse events associated with patient movement, allowances for off-floor movement should be carefully considered with input from risk management, physicians, nursing leadership, patient advocates, and hospital administration.

Additionally, unannounced movement off the floor may interfere with timely and efficient care by causing lapses in monitoring, such as cardiac telemetry,⁷ medication administration, and scheduled diagnostic tests. In these situations, the risks of patient absence from the floor are significant and may ultimately negate the benefits of continued hospitalization by compromising the central elements of patient care.

Patients’ requests to leave the hospital floor should be evaluated systemically and transparently to promote fair, high-value care. First, a request for liberalized movement should prompt physicians that the patient may no longer require hospitalization and may be ready for the transition to outpatient care.⁸ If the patient still requires inpatient care, then the medical practitioner should make a clinical determination if the patient is medically stable enough to leave their hospital floor. The provider should first identify when the liberalization of movement would be universally inappropriate, such as in patients who are physically unable to ambulate without posing significant harm to themselves. This includes an accidental fall (usually while walking⁵), which is one of the most commonly reported adverse events in an inpatient setting.⁹ Additionally, patients with significant cognitive impairments or those lacking in decision-making capacity may be restricted from leaving their floors unescorted, as they are at a higher risk of disorientation, falls, and death.¹⁰

In determining movement restrictions for patients in isolation, hospitals should refer to the existing guidelines on isolation precautions for the transmission of communicable infections^11,12 and neutropenic precautions.¹³ Additionally, movement restriction for patients who are isolated after screening positive for certain drug-resistant organisms (eg, methicillin-resistant Staphylococcus aureus and vancomycin-resistant enterococci) is controversial and should be evaluated based on the available medical evidence and standards.^14-16

When making a risk-benefit determination about movement, providers should also assess the intent and the potentially unmet needs behind the patient’s request. Patient-centered reasons for enhanced freedom of movement within the hospital include a desire for exercise, greater food choice, and visiting with loved ones, all of which can enable patients to manage the well-known inconveniences and stresses of hospitalization. In contrast, there may be concerns for other intentions behind leaving assigned floors based on the patient’s clinical history, such as the surreptitious use of illicit substances or attempts to elope from the hospital. Advising restriction of movement is justifiable if there is a significant concern for behavior that undermines the safe delivery of care. In patients with active substance use disorders, the appropriate treatment of pain or withdrawal symptoms may better address the patients’ unmet needs, but a lower threshold to restrict movement may be reasonable given the significant risks involved. However, given the widespread stigmatization of patients with substance use disorders,¹⁷ institutional policy and clinicians should adhere to systematic, transparent, and consistent risk assessments for all patients in order to minimize the potential for introducing or exacerbating disparities in care.

In order to work productively with admitted patients, strong practices honor patients’ autonomy by specifying when and how patients are informed of the institution’s expectations about and limitations to inpatient movement. For example, emergency room patients were less likely to elope when treatment expectations were established at the time of presentation by giving them information about wait times and triage protocol.¹⁸ Similarly, by preemptively discussing reasonable restrictions on movement as a part of informed consent for inpatient admission, physicians can establish patients’ expectations early in the admission process and foster a therapeutic alliance on the basis of the shared goals of safe and timely care.

Patients may request or even demand to leave the floor after a healthcare provider has determined that doing so would be unsafe and/or undermine the timely and efficient delivery of care. In these cases, shared decision-making (SDM) can help identify acceptable solutions within the identified constraints. SDM combines the physicians’ experience, expertise, and knowledge of medical evidence with patients’ values, needs, and preferences for care.¹⁹ If patients continue to request to leave the floor after the restriction has been communicated, physicians should discuss whether the current treatment plan should be renegotiated to include a relatively minor modification (eg, a change in the timing or route of administration of medication). If inpatient care cannot be provided safely within the patient’s preferences for movement and attempts to accommodate the patient’s preferences are unsuccessful, then a shift to discharge planning may be appropriate. A summary of this decision process is outlined in the Figure.

In mitigating the burdens of restriction on movement, hospitals may implement a range of options that address patients’ preferences while maintaining safety. Given the potential consequences of liberalized patient movement, it may be prudent to implement these safeguards as a compromise that addresses both the patients’ needs and the hospital’s concerns. These could include an escort for off-floor supervision, timed passes to leave the floor, or volunteers purchasing food for patients from the cafeteria. Creating open, supervised spaces within the hospital (eg, lounges) may also help provide the respite patients need, but in a safe and medically structured environment.

Returning to the introductory case example, we now present an alternative outcome in the context of the practices described above. On the morning of the scheduled TEE, a nurse noted that the patient was missing from his room. Before the staff began searching for the patient, they consulted the medical record which included the admission discussion and agreement to expectations for inpatient movement. The record also included an informed consent discussion indicating the minimal risks of leaving the floor, as the patient could ambulate independently and had no need for continuous monitoring. Finally, a physician’s order authorized the patient to be off the floor until 10 am. The patient returned to his room at 9:45 am and underwent a normal TEE, after which he was discharged home with outpatient follow-up.

The above scenario highlights the benefits of a comprehensive framework for patient movement practices that are transparent, fair, and systematic. Explicitly recognizing competing institutional and patient perspectives can prevent conflict and promote high-quality, safe, efficient, patient-centered care that only restricts the patient’s movement under specified and justifiable conditions. In developing strong hospital practices, institutions should refer to the relevant clinical and ethical standards and draw upon their institutional resources in risk management, clinical staff, and patient advocates.

Acknowledgments

The authors thank Dr. Neil Shapiro and Dr. David Chuquin for their constructive reviews of prior versions of this manuscript.

Disclosures

The authors have no financial conflicts of interest to disclose.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the U.S. Department of Veterans Affairs, the US Government, or the VA National Center for Ethics in Health Care.

A 58-year-old man with a remote history of endocarditis and no prior injection drug use was admitted to the inpatient medicine service with fever and concern for recurrent endocarditis. A transthoracic echocardiogram was unremarkable and the patient remained clinically stable. A transesophageal echocardiogram (TEE) was scheduled for the following morning, but during nursing rounds, the patient was missing from his room. Multiple staff members searched for the patient and eventually located him in the hospital lobby drinking a cup of coffee purchased from the cafeteria. Despite his opposition, he was escorted back to his room and advised to not leave the floor again. Later that day, the patient became frustrated and left the hospital before his scheduled TEE. He was subsequently lost to follow-up.

Patients are admitted to the hospital based upon a medical determination that the patient requires acute observation, evaluation, or treatment. Once admitted, healthcare providers may impose restrictions on the patient’s movement in the hospital, such as restrictions on leaving their assigned floor. Managing the movement of hospitalized patients poses significant challenges for the clinical staff because of the difficulty of providing a treatment environment that ensures safe and efficient delivery of care while promoting patients’ preferences for an unrestrictive environment that respects their independence.^1,2 Broad limits may make it easier for staff to care for patients and reduce concerns about liability, but they may also frustrate patients who may be medically, psychiatrically, and physically stable and do not require stringent monitoring (eg, completing a course of intravenous antibiotics or awaiting placement at outside facilities).

Although this issue has broad implications for patient safety and hospital liability, authoritative guidance and evidence-based literature are lacking. Without clear guidelines, healthcare staff members are likely to spend more time in managing each individual request to leave the floor because they do not have a systematic strategy for making fair and consistent decisions. Here, we describe the patient and institutional considerations when managing patient movement in the hospital. We refer to “patient movement” specifically as a patient’s choice to move to different locations within the hospital, but outside of their assigned room and/or floor. This does not include scheduled, supervised ambulation activities, such as physical therapy.

Practices that promote patient movement offer significant benefits and risks. Enhancing movement is likely to reduce the “physiologic disruption”³ of hospitalization while improving patients’ overall satisfaction and alignment with patient-centered care. Liberalized movement also promotes independence and ambulation that reduces the rate of physical deconditioning.⁴

Despite theoretical benefits, hospitals may be more concerned about adverse events related to patient movement, such as falls, the use of illicit substances, or elopement. Given that hospitals may be legally⁵ and financially responsible⁶ for adverse events associated with patient movement, allowances for off-floor movement should be carefully considered with input from risk management, physicians, nursing leadership, patient advocates, and hospital administration.

Additionally, unannounced movement off the floor may interfere with timely and efficient care by causing lapses in monitoring, such as cardiac telemetry,⁷ medication administration, and scheduled diagnostic tests. In these situations, the risks of patient absence from the floor are significant and may ultimately negate the benefits of continued hospitalization by compromising the central elements of patient care.

Patients’ requests to leave the hospital floor should be evaluated systemically and transparently to promote fair, high-value care. First, a request for liberalized movement should prompt physicians that the patient may no longer require hospitalization and may be ready for the transition to outpatient care.⁸ If the patient still requires inpatient care, then the medical practitioner should make a clinical determination if the patient is medically stable enough to leave their hospital floor. The provider should first identify when the liberalization of movement would be universally inappropriate, such as in patients who are physically unable to ambulate without posing significant harm to themselves. This includes an accidental fall (usually while walking⁵), which is one of the most commonly reported adverse events in an inpatient setting.⁹ Additionally, patients with significant cognitive impairments or those lacking in decision-making capacity may be restricted from leaving their floors unescorted, as they are at a higher risk of disorientation, falls, and death.¹⁰

In determining movement restrictions for patients in isolation, hospitals should refer to the existing guidelines on isolation precautions for the transmission of communicable infections^11,12 and neutropenic precautions.¹³ Additionally, movement restriction for patients who are isolated after screening positive for certain drug-resistant organisms (eg, methicillin-resistant Staphylococcus aureus and vancomycin-resistant enterococci) is controversial and should be evaluated based on the available medical evidence and standards.^14-16

When making a risk-benefit determination about movement, providers should also assess the intent and the potentially unmet needs behind the patient’s request. Patient-centered reasons for enhanced freedom of movement within the hospital include a desire for exercise, greater food choice, and visiting with loved ones, all of which can enable patients to manage the well-known inconveniences and stresses of hospitalization. In contrast, there may be concerns for other intentions behind leaving assigned floors based on the patient’s clinical history, such as the surreptitious use of illicit substances or attempts to elope from the hospital. Advising restriction of movement is justifiable if there is a significant concern for behavior that undermines the safe delivery of care. In patients with active substance use disorders, the appropriate treatment of pain or withdrawal symptoms may better address the patients’ unmet needs, but a lower threshold to restrict movement may be reasonable given the significant risks involved. However, given the widespread stigmatization of patients with substance use disorders,¹⁷ institutional policy and clinicians should adhere to systematic, transparent, and consistent risk assessments for all patients in order to minimize the potential for introducing or exacerbating disparities in care.

In order to work productively with admitted patients, strong practices honor patients’ autonomy by specifying when and how patients are informed of the institution’s expectations about and limitations to inpatient movement. For example, emergency room patients were less likely to elope when treatment expectations were established at the time of presentation by giving them information about wait times and triage protocol.¹⁸ Similarly, by preemptively discussing reasonable restrictions on movement as a part of informed consent for inpatient admission, physicians can establish patients’ expectations early in the admission process and foster a therapeutic alliance on the basis of the shared goals of safe and timely care.

Patients may request or even demand to leave the floor after a healthcare provider has determined that doing so would be unsafe and/or undermine the timely and efficient delivery of care. In these cases, shared decision-making (SDM) can help identify acceptable solutions within the identified constraints. SDM combines the physicians’ experience, expertise, and knowledge of medical evidence with patients’ values, needs, and preferences for care.¹⁹ If patients continue to request to leave the floor after the restriction has been communicated, physicians should discuss whether the current treatment plan should be renegotiated to include a relatively minor modification (eg, a change in the timing or route of administration of medication). If inpatient care cannot be provided safely within the patient’s preferences for movement and attempts to accommodate the patient’s preferences are unsuccessful, then a shift to discharge planning may be appropriate. A summary of this decision process is outlined in the Figure.

In mitigating the burdens of restriction on movement, hospitals may implement a range of options that address patients’ preferences while maintaining safety. Given the potential consequences of liberalized patient movement, it may be prudent to implement these safeguards as a compromise that addresses both the patients’ needs and the hospital’s concerns. These could include an escort for off-floor supervision, timed passes to leave the floor, or volunteers purchasing food for patients from the cafeteria. Creating open, supervised spaces within the hospital (eg, lounges) may also help provide the respite patients need, but in a safe and medically structured environment.

Returning to the introductory case example, we now present an alternative outcome in the context of the practices described above. On the morning of the scheduled TEE, a nurse noted that the patient was missing from his room. Before the staff began searching for the patient, they consulted the medical record which included the admission discussion and agreement to expectations for inpatient movement. The record also included an informed consent discussion indicating the minimal risks of leaving the floor, as the patient could ambulate independently and had no need for continuous monitoring. Finally, a physician’s order authorized the patient to be off the floor until 10 am. The patient returned to his room at 9:45 am and underwent a normal TEE, after which he was discharged home with outpatient follow-up.

The above scenario highlights the benefits of a comprehensive framework for patient movement practices that are transparent, fair, and systematic. Explicitly recognizing competing institutional and patient perspectives can prevent conflict and promote high-quality, safe, efficient, patient-centered care that only restricts the patient’s movement under specified and justifiable conditions. In developing strong hospital practices, institutions should refer to the relevant clinical and ethical standards and draw upon their institutional resources in risk management, clinical staff, and patient advocates.

Acknowledgments

The authors thank Dr. Neil Shapiro and Dr. David Chuquin for their constructive reviews of prior versions of this manuscript.

Disclosures

The authors have no financial conflicts of interest to disclose.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the U.S. Department of Veterans Affairs, the US Government, or the VA National Center for Ethics in Health Care.

A 58-year-old man with a remote history of endocarditis and no prior injection drug use was admitted to the inpatient medicine service with fever and concern for recurrent endocarditis. A transthoracic echocardiogram was unremarkable and the patient remained clinically stable. A transesophageal echocardiogram (TEE) was scheduled for the following morning, but during nursing rounds, the patient was missing from his room. Multiple staff members searched for the patient and eventually located him in the hospital lobby drinking a cup of coffee purchased from the cafeteria. Despite his opposition, he was escorted back to his room and advised to not leave the floor again. Later that day, the patient became frustrated and left the hospital before his scheduled TEE. He was subsequently lost to follow-up.

Patients are admitted to the hospital based upon a medical determination that the patient requires acute observation, evaluation, or treatment. Once admitted, healthcare providers may impose restrictions on the patient’s movement in the hospital, such as restrictions on leaving their assigned floor. Managing the movement of hospitalized patients poses significant challenges for the clinical staff because of the difficulty of providing a treatment environment that ensures safe and efficient delivery of care while promoting patients’ preferences for an unrestrictive environment that respects their independence.^1,2 Broad limits may make it easier for staff to care for patients and reduce concerns about liability, but they may also frustrate patients who may be medically, psychiatrically, and physically stable and do not require stringent monitoring (eg, completing a course of intravenous antibiotics or awaiting placement at outside facilities).

Although this issue has broad implications for patient safety and hospital liability, authoritative guidance and evidence-based literature are lacking. Without clear guidelines, healthcare staff members are likely to spend more time in managing each individual request to leave the floor because they do not have a systematic strategy for making fair and consistent decisions. Here, we describe the patient and institutional considerations when managing patient movement in the hospital. We refer to “patient movement” specifically as a patient’s choice to move to different locations within the hospital, but outside of their assigned room and/or floor. This does not include scheduled, supervised ambulation activities, such as physical therapy.

Practices that promote patient movement offer significant benefits and risks. Enhancing movement is likely to reduce the “physiologic disruption”³ of hospitalization while improving patients’ overall satisfaction and alignment with patient-centered care. Liberalized movement also promotes independence and ambulation that reduces the rate of physical deconditioning.⁴

Despite theoretical benefits, hospitals may be more concerned about adverse events related to patient movement, such as falls, the use of illicit substances, or elopement. Given that hospitals may be legally⁵ and financially responsible⁶ for adverse events associated with patient movement, allowances for off-floor movement should be carefully considered with input from risk management, physicians, nursing leadership, patient advocates, and hospital administration.

Additionally, unannounced movement off the floor may interfere with timely and efficient care by causing lapses in monitoring, such as cardiac telemetry,⁷ medication administration, and scheduled diagnostic tests. In these situations, the risks of patient absence from the floor are significant and may ultimately negate the benefits of continued hospitalization by compromising the central elements of patient care.

Patients’ requests to leave the hospital floor should be evaluated systemically and transparently to promote fair, high-value care. First, a request for liberalized movement should prompt physicians that the patient may no longer require hospitalization and may be ready for the transition to outpatient care.⁸ If the patient still requires inpatient care, then the medical practitioner should make a clinical determination if the patient is medically stable enough to leave their hospital floor. The provider should first identify when the liberalization of movement would be universally inappropriate, such as in patients who are physically unable to ambulate without posing significant harm to themselves. This includes an accidental fall (usually while walking⁵), which is one of the most commonly reported adverse events in an inpatient setting.⁹ Additionally, patients with significant cognitive impairments or those lacking in decision-making capacity may be restricted from leaving their floors unescorted, as they are at a higher risk of disorientation, falls, and death.¹⁰

In determining movement restrictions for patients in isolation, hospitals should refer to the existing guidelines on isolation precautions for the transmission of communicable infections^11,12 and neutropenic precautions.¹³ Additionally, movement restriction for patients who are isolated after screening positive for certain drug-resistant organisms (eg, methicillin-resistant Staphylococcus aureus and vancomycin-resistant enterococci) is controversial and should be evaluated based on the available medical evidence and standards.^14-16

When making a risk-benefit determination about movement, providers should also assess the intent and the potentially unmet needs behind the patient’s request. Patient-centered reasons for enhanced freedom of movement within the hospital include a desire for exercise, greater food choice, and visiting with loved ones, all of which can enable patients to manage the well-known inconveniences and stresses of hospitalization. In contrast, there may be concerns for other intentions behind leaving assigned floors based on the patient’s clinical history, such as the surreptitious use of illicit substances or attempts to elope from the hospital. Advising restriction of movement is justifiable if there is a significant concern for behavior that undermines the safe delivery of care. In patients with active substance use disorders, the appropriate treatment of pain or withdrawal symptoms may better address the patients’ unmet needs, but a lower threshold to restrict movement may be reasonable given the significant risks involved. However, given the widespread stigmatization of patients with substance use disorders,¹⁷ institutional policy and clinicians should adhere to systematic, transparent, and consistent risk assessments for all patients in order to minimize the potential for introducing or exacerbating disparities in care.

In order to work productively with admitted patients, strong practices honor patients’ autonomy by specifying when and how patients are informed of the institution’s expectations about and limitations to inpatient movement. For example, emergency room patients were less likely to elope when treatment expectations were established at the time of presentation by giving them information about wait times and triage protocol.¹⁸ Similarly, by preemptively discussing reasonable restrictions on movement as a part of informed consent for inpatient admission, physicians can establish patients’ expectations early in the admission process and foster a therapeutic alliance on the basis of the shared goals of safe and timely care.

Patients may request or even demand to leave the floor after a healthcare provider has determined that doing so would be unsafe and/or undermine the timely and efficient delivery of care. In these cases, shared decision-making (SDM) can help identify acceptable solutions within the identified constraints. SDM combines the physicians’ experience, expertise, and knowledge of medical evidence with patients’ values, needs, and preferences for care.¹⁹ If patients continue to request to leave the floor after the restriction has been communicated, physicians should discuss whether the current treatment plan should be renegotiated to include a relatively minor modification (eg, a change in the timing or route of administration of medication). If inpatient care cannot be provided safely within the patient’s preferences for movement and attempts to accommodate the patient’s preferences are unsuccessful, then a shift to discharge planning may be appropriate. A summary of this decision process is outlined in the Figure.

In mitigating the burdens of restriction on movement, hospitals may implement a range of options that address patients’ preferences while maintaining safety. Given the potential consequences of liberalized patient movement, it may be prudent to implement these safeguards as a compromise that addresses both the patients’ needs and the hospital’s concerns. These could include an escort for off-floor supervision, timed passes to leave the floor, or volunteers purchasing food for patients from the cafeteria. Creating open, supervised spaces within the hospital (eg, lounges) may also help provide the respite patients need, but in a safe and medically structured environment.

Returning to the introductory case example, we now present an alternative outcome in the context of the practices described above. On the morning of the scheduled TEE, a nurse noted that the patient was missing from his room. Before the staff began searching for the patient, they consulted the medical record which included the admission discussion and agreement to expectations for inpatient movement. The record also included an informed consent discussion indicating the minimal risks of leaving the floor, as the patient could ambulate independently and had no need for continuous monitoring. Finally, a physician’s order authorized the patient to be off the floor until 10 am. The patient returned to his room at 9:45 am and underwent a normal TEE, after which he was discharged home with outpatient follow-up.

The above scenario highlights the benefits of a comprehensive framework for patient movement practices that are transparent, fair, and systematic. Explicitly recognizing competing institutional and patient perspectives can prevent conflict and promote high-quality, safe, efficient, patient-centered care that only restricts the patient’s movement under specified and justifiable conditions. In developing strong hospital practices, institutions should refer to the relevant clinical and ethical standards and draw upon their institutional resources in risk management, clinical staff, and patient advocates.

Acknowledgments

The authors thank Dr. Neil Shapiro and Dr. David Chuquin for their constructive reviews of prior versions of this manuscript.

Disclosures

The authors have no financial conflicts of interest to disclose.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the U.S. Department of Veterans Affairs, the US Government, or the VA National Center for Ethics in Health Care.

Sepsis and heart failure are two common, costly, and deadly conditions. Among hospitalized Medicare patients, these conditions rank as the first and second most frequent principal diagnoses accounting for over $33 billion in spending across all payers.¹ One-third to one-half of all hospital deaths are estimated to occur in patients with sepsis,² and heart failure is listed as a contributing factor in over 10% of deaths in the United States.³

Previous research shows that evidence-based care decisions can impact the outcomes for these patients. For example, sepsis patients receiving intravenous fluids, blood cultures, broad-spectrum antibiotics, and lactate measurement within three hours of presentation have lower mortality rates.⁴ In heart failure, key interventions such as the appropriate use of ACE inhibitors, beta blockers, and referral to disease management programs reduce morbidity and mortality.⁵

However, rapid dissemination and adoption of evidence-based guidelines remain a challenge.^6,7 Policy makers have introduced incentives and penalties to support adoption, with varying levels of success. After four years of Centers for Medicare and Medicaid Services (CMS) penalties for hospitals with excess heart failure readmissions, only 21% performed well enough to avoid a penalty in 2017.⁸ CMS has been tracking sepsis bundle adherence as a core measure, but the rate in 2018 sat at just 54%.⁹ It is clear that new solutions are needed.¹⁰

AdventHealth (formerly Adventist Health System) is a growing, faith-based health system with hospitals across nine states. AdventHealth is a national leader in quality, safety, and patient satisfaction but is not immune to the challenges of delivering consistent, evidence-based care across an extensive network. To accelerate system-wide practice change, AdventHealth’s Office of Clinical Excellence (OCE) partnered with QURE Healthcare and Premier, Inc., to implement a physician engagement and care standardization collaboration involving nearly 100 hospitalists at eight facilities across five states.

This paper describes the results of the Adventist QURE Quality Project (AQQP), which used QURE’s validated, simulation-based measurement and feedback approach to engage hospitalists and standardize evidence-based practices for patients with sepsis and heart failure. We documented specific areas of variation identified in the simulations, how those practices changed through serial feedback, and the impact of those changes on real-world outcomes and costs.

Setting

AdventHealth has its headquarters in Altamonte Springs, Florida. It has facilities in nine states, which includes 48 hospitals. The OCE is comprised of physician leaders, project managers, and data analysts who sponsored the project from July 2016 through July 2018.

Study Participants

AdventHealth hospitals were invited to enroll their hospitalists in AQQP; eight AdventHealth hospitals across five states, representing 91 physicians and 16 nurse practitioners/physician’s assistants (APPs), agreed to participate. Participants included both AdventHealth-employed providers and contracted hospitalist groups. Provider participation was voluntary and not tied to financial incentives; however, participants received Continuing Medical Education credit and, if applicable, Maintenance of Certification points through the American Board of Internal Medicine.

Quasi-experimental Design

We used AdventHealth hospitals not participating in AQQP as a quasi-experimental control group. We leveraged this to measure the impact of concurrent secular effects, such as order sets and other system-wide training, that could also improve practice and outcomes in our study.

Study Objectives and Approach

The explicit goals of AQQP were to (1) measure how sepsis and heart failure patients are cared for across AdventHealth using Clinical Performance and Value (CPV) case simulations, (2) provide a forum for hospitalists to discuss clinical variation, and (3) reduce unneeded variation to improve quality and reduce cost. QURE developed 12 CPV simulated patient cases (six sepsis and six heart failure cases) with case-specific evidenced-based scoring criteria tied to national and Advent­Health evidence-based guidelines. AdventHealth order sets were embedded in the cases and accessible by participants as they cared for their patients.

CPV vignettes are simulated patient cases administered online, and have been validated as an accurate and responsive measure of clinical decision-making in both ambulatory^11-13 and inpatient settings.^14,15 Cases take 20-30 minutes each to complete and simulate a typical clinical encounter: taking the medical history, performing a physical examination, ordering tests, making the diagnosis, implementing initial treatment, and outlining a follow-up plan. Each case has predefined, evidence-based scoring criteria for each care domain. Cases and scoring criteria were reviewed by AdventHealth hospitalist program leaders and physician leaders in OCE. Provider responses were double-scored by trained physician abstractors. Scores range from 0%-100%, with higher scores reflecting greater alignment with best practice recommendations.

In each round of the project, AQQP participants completed two CPV cases, received personalized online feedback reports on their care decisions, and met (at the various sites and via web conference) for a facilitated group discussion on areas of high group variation. The personal feedback reports included the participant’s case score compared to the group average, a list of high-priority personalized improvement opportunities, a summary of the cost of unneeded care items, and links to relevant references. The group discussions focused on six items of high variation. Six total rounds of CPV measurement and feedback were held, one every four months.

At the study’s conclusion, we administered a brief satisfaction survey, asking providers to rate various aspects of the project on a five-point Likert scale.

Data

The study used two primary data sources: (1) care decisions made in the CPV simulated cases and (2) patient-level utilization data from Premier Inc.’s QualityAdvisor^TM (QA) data system. QA integrates quality, safety, and financial data from AdventHealth’s electronic medical record, claims data, charge master, and other resources. QualityAdvisor also calculates expected performance for critical measures, including cost per case and length of stay (LOS), based on a proprietary algorithm, which uses DRG classification, severity-of-illness, risk-of-mortality, and other patient risk factors. We pulled patient-level observed and expected data from AQQP qualifying physicians, defined as physicians participating in a majority of CPV measurement rounds. Of the 107 total hospitalists who participated, six providers did not participate in enough CPV rounds, and 22 providers left AdventHealth and could not be included in a patient-level impact analysis. These providers were replaced with 21 new hospitalists who were enrolled in the study and included in the CPV analysis but who did not have patient-level data before AQQP enrollment. Overall, 58 providers met the qualifying criteria to be included in the impact analysis. We compared their performance to a group of 96 hospitalists at facilities that were not participating in the project. Comparator facilities were selected based on quantitative measures of size and demographic matching the AQQP-facilities ensuring that both sets of hospitals (comparator and AQQP) exhibited similar levels of engagement with Advent- Health quality activities such as quality dashboard performance and order set usage. Baseline patient-level cost and LOS data covered from October 2015 to June 2016 and were re-measured annually throughout the project, from July 2016 to June 2018.

Statistical Analyses

We analyzed three primary outcomes: (1) general CPV-measured improvements in each round (scored against evidence-based scoring criteria); (2) disease-specific CPV improvements over each round; and (3) changes in patient-level outcomes and economic savings among AdventHealth pneumonia/sepsis and heart failure patients from the aforementioned improvements. We used Student’s t-test to analyze continuous outcome variables (including CPV, cost of care, and length of stay data) and Fisher’s exact test for binary outcome data. All statistical analyses were performed using Stata 14.2 (StataCorp LLC, College Station, Texas).

Baseline Characteristics and Assessment

A total of 107 AdventHealth hospitalists participated in this study (Appendix Table 1). 78.1% of these providers rated the organization’s focus on quality and lowering unnecessary costs as either “good” or “excellent,” but 78.8% also reported that variation in care provided by the group was “moderate” to “very high”.

At baseline, we observed high variability in the care of pneumonia patients with sepsis (pneumonia/sepsis) and heart failure patients as measured by the care decisions obtained in the CPV cases. The overall quality score, which is a weighted average across all domains, averaged 61.9% ± 10.5% for the group (Table 1). Disaggregating scores by condition, we found an average overall score of 59.4% ± 10.9% for pneumonia/sepsis and 64.4% ± 9.4% for heart failure. The diagnosis and treatment domains, which require the most clinical judgment, had the lowest average domain scores of 53.4% ± 20.9% and 51.6% ± 15.1%, respectively.

Changes in CPV Scores

To determine the impact of serial measurement and feedback, we compared performance in the first two rounds of the project with the last two rounds. We found that overall CPV quality scores showed a 4.8%-point absolute improvement (P < .001; Table 1). We saw improvements in all care domains, and those increases were significant in all but the workup (P = .470); the most significant increase was in diagnostic accuracy (+19.1%; P < .001).

By condition, scores showed similar, statistically significant overall improvements: +4.4%-points for pneumonia/sepsis (P = .001) and +5.5%-points for heart failure (P < .001) driven by increases in the diagnosis and treatment domains. For example, providers increased appropriate identification of HF severity by 21.5%-points (P < .001) and primary diagnosis of pneumonia/sepsis by 3.6%-points (P = .385).

In the treatment domain, which included clinical decisions related to initial management and follow-up care, there were several specific improvements. For HF, we found that performing all the essential treatment elements—prescribing diuretics, ACE inhibitors and beta blockers for appropriate patients—improved by 13.9%-points (P = .038); ordering VTE prophylaxis increased more than threefold, from 16.6% to 51.0% (P < .001; Table 2). For pneumonia/sepsis patients, absolute adherence to all four elements of the 3-hour sepsis bundle improved by 11.7%-points (P = .034). We also saw a decrease in low-value diagnostic workup items for patient cases in which the guidelines suggest they are not needed, such as urinary antigen testing, which declined by 14.6%-points (P = .001) and sputum cultures, which declined 26.4%-points (P = .004). In addition, outlining an evidence-based discharge plan including a follow-up visit, patient education and medication reconciliation improved, especially for pneumonia/sepsis patients by 24.3%-points (P < .001).

Patient-Level Impact on LOS and Cost Per Case

We took advantage of the quasi-experimental design, in which only a portion of AdventHealth facilities participated in the project, to compare patient-level results from AQQP-participating physicians against the engagement-matched cohort of hospitalists at nonparticipating AdventHealth facilities. We adjusted for potential differences in patient-level case mix between the two groups by comparing the observed/expected (O/E) LOS and cost per case ratios for pneumonia/sepsis and heart failure patients.

At baseline, AQQP-hospitalists performed better on geometric LOS versus the comparator group (O/E of 1.13 vs 1.22; P = .006) but at about the same on cost per case (O/E of 1.16 vs 1.14; P = .390). Throughout the project, as patient volumes and expected per patient costs rose for both groups, O/E ratios improved among both AQQP and non-AQQP providers.

To set apart the contribution of system-wide improvements from the AQQP project-specific impacts, we applied the O/E improvement rates seen in the comparator group to the AQQP group baseline performance. We then compared that to the actual changes seen in the AQQP throughout the project to see if there was any additional benefit from the simulation measurement and feedback (Figure).

Provider Satisfaction

At the project conclusion, we administered a brief survey. Participants were asked to rate aspects of the project (a five-point Likert scale with five being the highest), and 24 responded. The mean ratings of the relevance of the project to their practice and the overall quality of the material were 4.5 and 4.2, respectively. Providers found the individual feedback reports (3.9) slightly more helpful than the webcast group discussions (3.7; Appendix Table 2 ).

As health systems expand, the opportunity to standardize clinical practice within a system has the potential to enhance patient care and lower costs. However, achieving these goals is challenging when providers are dispersed across geographically separated sites and clinical decision-making is difficult to measure in a standardized way.^16,17 We brought together over 100 physicians and APPs from eight different-sized hospitals in five different states to prospectively determine if we could improve care using a standardized measurement and feedback system. At baseline, we found that care varied dramatically among providers. Care varied in terms of diagnostic accuracy and treatment, which directly relate to care quality and outcomes.⁴ After serial measurement and feedback, we saw reductions in unnecessary testing, more guideline-based treatment decisions, and better discharge planning in the clinical vignettes.

We confirmed that changes in CPV-measured practice translated into lower costs and shorter LOS at the patient level. We further validated the improvements through a quasi-experimental design that compared these changes to those at nonparticipating AdventHealth facilities. We saw more significant cost reductions and decreases in LOS in the simulation-based measurement and feedback cohort with the biggest impact early on. The overall savings to the system, attributable specifically to the AQQP approach, is estimated to be $2.4 million.

One advantage of the online case simulation approach is the ability to bring geographically remote sites together in a shared quality-of-care discussion. The interventions specifically sought to remove barriers between facilities. For example, individual feedback reports allowed providers to see how they compare with providers at other AdventHealth facilities and webcast results discussions enable providers across facilities to discuss specific care decisions.

There were several limitations to the study. While the quasi-experimental design allowed us to make informative comparisons between AQQP-participating facilities and nonparticipating facilities, the assignments were not random, and participants were generally from higher performing hospital medicine groups. The determination of secular versus CPV-related improvement is confounded by other system improvement initiatives that may have impacted cost and LOS results. This is mitigated by the observation that facilities that opted to participate performed better at baseline in risk-adjusted LOS but slightly worse in cost per case, indicating that baseline differences were not dramatic. While both groups improved over time, the QURE measurement and feedback approach led to larger and more rapid gains than those seen in the comparator group. However, we could not exclude the potential that project participation at the site level was biased to those groups disposed to performance improvement. In addition, our patient-level data analysis was limited to the metrics available and did not allow us to directly compare patient-level performance across the plethora of clinically relevant CPV data that showed improvement. Our inpatient cost per case analysis showed significant savings for the system but did not include all potentially favorable economic impacts such as lower follow-up care costs for patients, more accurate reimbursement through better coding or fewer lost days of productivity.

With continued consolidation in healthcare and broader health systems spanning multiple geographies, new tools are needed to support standardized, evidence-based care across sites. This standardization is especially important, both clinically and financially, for high-volume, high-cost diseases such as sepsis and heart failure. However, changing practice cannot happen without collaborative engagement with providers. Standardized patient vignettes are an opportunity to measure and provide feedback in a systematic way that engages providers and is particularly well-suited to large systems and common clinical conditions. This analysis, from a real-world study, shows that an approach that standardizes care and lowers costs may be particularly helpful for large systems needing to bring disparate sites together as they concurrently move toward value-based payment.

Disclosures

QURE, LLC, whose intellectual property was used to prepare the cases and collect the data, was contracted by AdventHealth. Otherwise, any of the study authors report no potential conflicts to disclose.

Funding

This work was funded by a contract between AdventHealth (formerly Adventist Health System) and QURE, LLC.

Sepsis and heart failure are two common, costly, and deadly conditions. Among hospitalized Medicare patients, these conditions rank as the first and second most frequent principal diagnoses accounting for over $33 billion in spending across all payers.¹ One-third to one-half of all hospital deaths are estimated to occur in patients with sepsis,² and heart failure is listed as a contributing factor in over 10% of deaths in the United States.³

Previous research shows that evidence-based care decisions can impact the outcomes for these patients. For example, sepsis patients receiving intravenous fluids, blood cultures, broad-spectrum antibiotics, and lactate measurement within three hours of presentation have lower mortality rates.⁴ In heart failure, key interventions such as the appropriate use of ACE inhibitors, beta blockers, and referral to disease management programs reduce morbidity and mortality.⁵

However, rapid dissemination and adoption of evidence-based guidelines remain a challenge.^6,7 Policy makers have introduced incentives and penalties to support adoption, with varying levels of success. After four years of Centers for Medicare and Medicaid Services (CMS) penalties for hospitals with excess heart failure readmissions, only 21% performed well enough to avoid a penalty in 2017.⁸ CMS has been tracking sepsis bundle adherence as a core measure, but the rate in 2018 sat at just 54%.⁹ It is clear that new solutions are needed.¹⁰

AdventHealth (formerly Adventist Health System) is a growing, faith-based health system with hospitals across nine states. AdventHealth is a national leader in quality, safety, and patient satisfaction but is not immune to the challenges of delivering consistent, evidence-based care across an extensive network. To accelerate system-wide practice change, AdventHealth’s Office of Clinical Excellence (OCE) partnered with QURE Healthcare and Premier, Inc., to implement a physician engagement and care standardization collaboration involving nearly 100 hospitalists at eight facilities across five states.

This paper describes the results of the Adventist QURE Quality Project (AQQP), which used QURE’s validated, simulation-based measurement and feedback approach to engage hospitalists and standardize evidence-based practices for patients with sepsis and heart failure. We documented specific areas of variation identified in the simulations, how those practices changed through serial feedback, and the impact of those changes on real-world outcomes and costs.

Setting

AdventHealth has its headquarters in Altamonte Springs, Florida. It has facilities in nine states, which includes 48 hospitals. The OCE is comprised of physician leaders, project managers, and data analysts who sponsored the project from July 2016 through July 2018.

Study Participants

AdventHealth hospitals were invited to enroll their hospitalists in AQQP; eight AdventHealth hospitals across five states, representing 91 physicians and 16 nurse practitioners/physician’s assistants (APPs), agreed to participate. Participants included both AdventHealth-employed providers and contracted hospitalist groups. Provider participation was voluntary and not tied to financial incentives; however, participants received Continuing Medical Education credit and, if applicable, Maintenance of Certification points through the American Board of Internal Medicine.

Quasi-experimental Design

We used AdventHealth hospitals not participating in AQQP as a quasi-experimental control group. We leveraged this to measure the impact of concurrent secular effects, such as order sets and other system-wide training, that could also improve practice and outcomes in our study.

Study Objectives and Approach

The explicit goals of AQQP were to (1) measure how sepsis and heart failure patients are cared for across AdventHealth using Clinical Performance and Value (CPV) case simulations, (2) provide a forum for hospitalists to discuss clinical variation, and (3) reduce unneeded variation to improve quality and reduce cost. QURE developed 12 CPV simulated patient cases (six sepsis and six heart failure cases) with case-specific evidenced-based scoring criteria tied to national and Advent­Health evidence-based guidelines. AdventHealth order sets were embedded in the cases and accessible by participants as they cared for their patients.

CPV vignettes are simulated patient cases administered online, and have been validated as an accurate and responsive measure of clinical decision-making in both ambulatory^11-13 and inpatient settings.^14,15 Cases take 20-30 minutes each to complete and simulate a typical clinical encounter: taking the medical history, performing a physical examination, ordering tests, making the diagnosis, implementing initial treatment, and outlining a follow-up plan. Each case has predefined, evidence-based scoring criteria for each care domain. Cases and scoring criteria were reviewed by AdventHealth hospitalist program leaders and physician leaders in OCE. Provider responses were double-scored by trained physician abstractors. Scores range from 0%-100%, with higher scores reflecting greater alignment with best practice recommendations.

In each round of the project, AQQP participants completed two CPV cases, received personalized online feedback reports on their care decisions, and met (at the various sites and via web conference) for a facilitated group discussion on areas of high group variation. The personal feedback reports included the participant’s case score compared to the group average, a list of high-priority personalized improvement opportunities, a summary of the cost of unneeded care items, and links to relevant references. The group discussions focused on six items of high variation. Six total rounds of CPV measurement and feedback were held, one every four months.

At the study’s conclusion, we administered a brief satisfaction survey, asking providers to rate various aspects of the project on a five-point Likert scale.

Data

The study used two primary data sources: (1) care decisions made in the CPV simulated cases and (2) patient-level utilization data from Premier Inc.’s QualityAdvisor^TM (QA) data system. QA integrates quality, safety, and financial data from AdventHealth’s electronic medical record, claims data, charge master, and other resources. QualityAdvisor also calculates expected performance for critical measures, including cost per case and length of stay (LOS), based on a proprietary algorithm, which uses DRG classification, severity-of-illness, risk-of-mortality, and other patient risk factors. We pulled patient-level observed and expected data from AQQP qualifying physicians, defined as physicians participating in a majority of CPV measurement rounds. Of the 107 total hospitalists who participated, six providers did not participate in enough CPV rounds, and 22 providers left AdventHealth and could not be included in a patient-level impact analysis. These providers were replaced with 21 new hospitalists who were enrolled in the study and included in the CPV analysis but who did not have patient-level data before AQQP enrollment. Overall, 58 providers met the qualifying criteria to be included in the impact analysis. We compared their performance to a group of 96 hospitalists at facilities that were not participating in the project. Comparator facilities were selected based on quantitative measures of size and demographic matching the AQQP-facilities ensuring that both sets of hospitals (comparator and AQQP) exhibited similar levels of engagement with Advent- Health quality activities such as quality dashboard performance and order set usage. Baseline patient-level cost and LOS data covered from October 2015 to June 2016 and were re-measured annually throughout the project, from July 2016 to June 2018.

Statistical Analyses

We analyzed three primary outcomes: (1) general CPV-measured improvements in each round (scored against evidence-based scoring criteria); (2) disease-specific CPV improvements over each round; and (3) changes in patient-level outcomes and economic savings among AdventHealth pneumonia/sepsis and heart failure patients from the aforementioned improvements. We used Student’s t-test to analyze continuous outcome variables (including CPV, cost of care, and length of stay data) and Fisher’s exact test for binary outcome data. All statistical analyses were performed using Stata 14.2 (StataCorp LLC, College Station, Texas).

Baseline Characteristics and Assessment

A total of 107 AdventHealth hospitalists participated in this study (Appendix Table 1). 78.1% of these providers rated the organization’s focus on quality and lowering unnecessary costs as either “good” or “excellent,” but 78.8% also reported that variation in care provided by the group was “moderate” to “very high”.

At baseline, we observed high variability in the care of pneumonia patients with sepsis (pneumonia/sepsis) and heart failure patients as measured by the care decisions obtained in the CPV cases. The overall quality score, which is a weighted average across all domains, averaged 61.9% ± 10.5% for the group (Table 1). Disaggregating scores by condition, we found an average overall score of 59.4% ± 10.9% for pneumonia/sepsis and 64.4% ± 9.4% for heart failure. The diagnosis and treatment domains, which require the most clinical judgment, had the lowest average domain scores of 53.4% ± 20.9% and 51.6% ± 15.1%, respectively.

Changes in CPV Scores

To determine the impact of serial measurement and feedback, we compared performance in the first two rounds of the project with the last two rounds. We found that overall CPV quality scores showed a 4.8%-point absolute improvement (P < .001; Table 1). We saw improvements in all care domains, and those increases were significant in all but the workup (P = .470); the most significant increase was in diagnostic accuracy (+19.1%; P < .001).

By condition, scores showed similar, statistically significant overall improvements: +4.4%-points for pneumonia/sepsis (P = .001) and +5.5%-points for heart failure (P < .001) driven by increases in the diagnosis and treatment domains. For example, providers increased appropriate identification of HF severity by 21.5%-points (P < .001) and primary diagnosis of pneumonia/sepsis by 3.6%-points (P = .385).

In the treatment domain, which included clinical decisions related to initial management and follow-up care, there were several specific improvements. For HF, we found that performing all the essential treatment elements—prescribing diuretics, ACE inhibitors and beta blockers for appropriate patients—improved by 13.9%-points (P = .038); ordering VTE prophylaxis increased more than threefold, from 16.6% to 51.0% (P < .001; Table 2). For pneumonia/sepsis patients, absolute adherence to all four elements of the 3-hour sepsis bundle improved by 11.7%-points (P = .034). We also saw a decrease in low-value diagnostic workup items for patient cases in which the guidelines suggest they are not needed, such as urinary antigen testing, which declined by 14.6%-points (P = .001) and sputum cultures, which declined 26.4%-points (P = .004). In addition, outlining an evidence-based discharge plan including a follow-up visit, patient education and medication reconciliation improved, especially for pneumonia/sepsis patients by 24.3%-points (P < .001).

Patient-Level Impact on LOS and Cost Per Case

We took advantage of the quasi-experimental design, in which only a portion of AdventHealth facilities participated in the project, to compare patient-level results from AQQP-participating physicians against the engagement-matched cohort of hospitalists at nonparticipating AdventHealth facilities. We adjusted for potential differences in patient-level case mix between the two groups by comparing the observed/expected (O/E) LOS and cost per case ratios for pneumonia/sepsis and heart failure patients.

At baseline, AQQP-hospitalists performed better on geometric LOS versus the comparator group (O/E of 1.13 vs 1.22; P = .006) but at about the same on cost per case (O/E of 1.16 vs 1.14; P = .390). Throughout the project, as patient volumes and expected per patient costs rose for both groups, O/E ratios improved among both AQQP and non-AQQP providers.

To set apart the contribution of system-wide improvements from the AQQP project-specific impacts, we applied the O/E improvement rates seen in the comparator group to the AQQP group baseline performance. We then compared that to the actual changes seen in the AQQP throughout the project to see if there was any additional benefit from the simulation measurement and feedback (Figure).

Provider Satisfaction

At the project conclusion, we administered a brief survey. Participants were asked to rate aspects of the project (a five-point Likert scale with five being the highest), and 24 responded. The mean ratings of the relevance of the project to their practice and the overall quality of the material were 4.5 and 4.2, respectively. Providers found the individual feedback reports (3.9) slightly more helpful than the webcast group discussions (3.7; Appendix Table 2 ).

As health systems expand, the opportunity to standardize clinical practice within a system has the potential to enhance patient care and lower costs. However, achieving these goals is challenging when providers are dispersed across geographically separated sites and clinical decision-making is difficult to measure in a standardized way.^16,17 We brought together over 100 physicians and APPs from eight different-sized hospitals in five different states to prospectively determine if we could improve care using a standardized measurement and feedback system. At baseline, we found that care varied dramatically among providers. Care varied in terms of diagnostic accuracy and treatment, which directly relate to care quality and outcomes.⁴ After serial measurement and feedback, we saw reductions in unnecessary testing, more guideline-based treatment decisions, and better discharge planning in the clinical vignettes.

We confirmed that changes in CPV-measured practice translated into lower costs and shorter LOS at the patient level. We further validated the improvements through a quasi-experimental design that compared these changes to those at nonparticipating AdventHealth facilities. We saw more significant cost reductions and decreases in LOS in the simulation-based measurement and feedback cohort with the biggest impact early on. The overall savings to the system, attributable specifically to the AQQP approach, is estimated to be $2.4 million.

One advantage of the online case simulation approach is the ability to bring geographically remote sites together in a shared quality-of-care discussion. The interventions specifically sought to remove barriers between facilities. For example, individual feedback reports allowed providers to see how they compare with providers at other AdventHealth facilities and webcast results discussions enable providers across facilities to discuss specific care decisions.

There were several limitations to the study. While the quasi-experimental design allowed us to make informative comparisons between AQQP-participating facilities and nonparticipating facilities, the assignments were not random, and participants were generally from higher performing hospital medicine groups. The determination of secular versus CPV-related improvement is confounded by other system improvement initiatives that may have impacted cost and LOS results. This is mitigated by the observation that facilities that opted to participate performed better at baseline in risk-adjusted LOS but slightly worse in cost per case, indicating that baseline differences were not dramatic. While both groups improved over time, the QURE measurement and feedback approach led to larger and more rapid gains than those seen in the comparator group. However, we could not exclude the potential that project participation at the site level was biased to those groups disposed to performance improvement. In addition, our patient-level data analysis was limited to the metrics available and did not allow us to directly compare patient-level performance across the plethora of clinically relevant CPV data that showed improvement. Our inpatient cost per case analysis showed significant savings for the system but did not include all potentially favorable economic impacts such as lower follow-up care costs for patients, more accurate reimbursement through better coding or fewer lost days of productivity.

With continued consolidation in healthcare and broader health systems spanning multiple geographies, new tools are needed to support standardized, evidence-based care across sites. This standardization is especially important, both clinically and financially, for high-volume, high-cost diseases such as sepsis and heart failure. However, changing practice cannot happen without collaborative engagement with providers. Standardized patient vignettes are an opportunity to measure and provide feedback in a systematic way that engages providers and is particularly well-suited to large systems and common clinical conditions. This analysis, from a real-world study, shows that an approach that standardizes care and lowers costs may be particularly helpful for large systems needing to bring disparate sites together as they concurrently move toward value-based payment.

Disclosures

QURE, LLC, whose intellectual property was used to prepare the cases and collect the data, was contracted by AdventHealth. Otherwise, any of the study authors report no potential conflicts to disclose.

Funding

This work was funded by a contract between AdventHealth (formerly Adventist Health System) and QURE, LLC.

Sepsis and heart failure are two common, costly, and deadly conditions. Among hospitalized Medicare patients, these conditions rank as the first and second most frequent principal diagnoses accounting for over $33 billion in spending across all payers.¹ One-third to one-half of all hospital deaths are estimated to occur in patients with sepsis,² and heart failure is listed as a contributing factor in over 10% of deaths in the United States.³

Previous research shows that evidence-based care decisions can impact the outcomes for these patients. For example, sepsis patients receiving intravenous fluids, blood cultures, broad-spectrum antibiotics, and lactate measurement within three hours of presentation have lower mortality rates.⁴ In heart failure, key interventions such as the appropriate use of ACE inhibitors, beta blockers, and referral to disease management programs reduce morbidity and mortality.⁵

However, rapid dissemination and adoption of evidence-based guidelines remain a challenge.^6,7 Policy makers have introduced incentives and penalties to support adoption, with varying levels of success. After four years of Centers for Medicare and Medicaid Services (CMS) penalties for hospitals with excess heart failure readmissions, only 21% performed well enough to avoid a penalty in 2017.⁸ CMS has been tracking sepsis bundle adherence as a core measure, but the rate in 2018 sat at just 54%.⁹ It is clear that new solutions are needed.¹⁰

AdventHealth (formerly Adventist Health System) is a growing, faith-based health system with hospitals across nine states. AdventHealth is a national leader in quality, safety, and patient satisfaction but is not immune to the challenges of delivering consistent, evidence-based care across an extensive network. To accelerate system-wide practice change, AdventHealth’s Office of Clinical Excellence (OCE) partnered with QURE Healthcare and Premier, Inc., to implement a physician engagement and care standardization collaboration involving nearly 100 hospitalists at eight facilities across five states.

This paper describes the results of the Adventist QURE Quality Project (AQQP), which used QURE’s validated, simulation-based measurement and feedback approach to engage hospitalists and standardize evidence-based practices for patients with sepsis and heart failure. We documented specific areas of variation identified in the simulations, how those practices changed through serial feedback, and the impact of those changes on real-world outcomes and costs.

Setting

AdventHealth has its headquarters in Altamonte Springs, Florida. It has facilities in nine states, which includes 48 hospitals. The OCE is comprised of physician leaders, project managers, and data analysts who sponsored the project from July 2016 through July 2018.

Study Participants

AdventHealth hospitals were invited to enroll their hospitalists in AQQP; eight AdventHealth hospitals across five states, representing 91 physicians and 16 nurse practitioners/physician’s assistants (APPs), agreed to participate. Participants included both AdventHealth-employed providers and contracted hospitalist groups. Provider participation was voluntary and not tied to financial incentives; however, participants received Continuing Medical Education credit and, if applicable, Maintenance of Certification points through the American Board of Internal Medicine.

Quasi-experimental Design

We used AdventHealth hospitals not participating in AQQP as a quasi-experimental control group. We leveraged this to measure the impact of concurrent secular effects, such as order sets and other system-wide training, that could also improve practice and outcomes in our study.

Study Objectives and Approach

The explicit goals of AQQP were to (1) measure how sepsis and heart failure patients are cared for across AdventHealth using Clinical Performance and Value (CPV) case simulations, (2) provide a forum for hospitalists to discuss clinical variation, and (3) reduce unneeded variation to improve quality and reduce cost. QURE developed 12 CPV simulated patient cases (six sepsis and six heart failure cases) with case-specific evidenced-based scoring criteria tied to national and Advent­Health evidence-based guidelines. AdventHealth order sets were embedded in the cases and accessible by participants as they cared for their patients.

CPV vignettes are simulated patient cases administered online, and have been validated as an accurate and responsive measure of clinical decision-making in both ambulatory^11-13 and inpatient settings.^14,15 Cases take 20-30 minutes each to complete and simulate a typical clinical encounter: taking the medical history, performing a physical examination, ordering tests, making the diagnosis, implementing initial treatment, and outlining a follow-up plan. Each case has predefined, evidence-based scoring criteria for each care domain. Cases and scoring criteria were reviewed by AdventHealth hospitalist program leaders and physician leaders in OCE. Provider responses were double-scored by trained physician abstractors. Scores range from 0%-100%, with higher scores reflecting greater alignment with best practice recommendations.

In each round of the project, AQQP participants completed two CPV cases, received personalized online feedback reports on their care decisions, and met (at the various sites and via web conference) for a facilitated group discussion on areas of high group variation. The personal feedback reports included the participant’s case score compared to the group average, a list of high-priority personalized improvement opportunities, a summary of the cost of unneeded care items, and links to relevant references. The group discussions focused on six items of high variation. Six total rounds of CPV measurement and feedback were held, one every four months.

At the study’s conclusion, we administered a brief satisfaction survey, asking providers to rate various aspects of the project on a five-point Likert scale.

Data

The study used two primary data sources: (1) care decisions made in the CPV simulated cases and (2) patient-level utilization data from Premier Inc.’s QualityAdvisor^TM (QA) data system. QA integrates quality, safety, and financial data from AdventHealth’s electronic medical record, claims data, charge master, and other resources. QualityAdvisor also calculates expected performance for critical measures, including cost per case and length of stay (LOS), based on a proprietary algorithm, which uses DRG classification, severity-of-illness, risk-of-mortality, and other patient risk factors. We pulled patient-level observed and expected data from AQQP qualifying physicians, defined as physicians participating in a majority of CPV measurement rounds. Of the 107 total hospitalists who participated, six providers did not participate in enough CPV rounds, and 22 providers left AdventHealth and could not be included in a patient-level impact analysis. These providers were replaced with 21 new hospitalists who were enrolled in the study and included in the CPV analysis but who did not have patient-level data before AQQP enrollment. Overall, 58 providers met the qualifying criteria to be included in the impact analysis. We compared their performance to a group of 96 hospitalists at facilities that were not participating in the project. Comparator facilities were selected based on quantitative measures of size and demographic matching the AQQP-facilities ensuring that both sets of hospitals (comparator and AQQP) exhibited similar levels of engagement with Advent- Health quality activities such as quality dashboard performance and order set usage. Baseline patient-level cost and LOS data covered from October 2015 to June 2016 and were re-measured annually throughout the project, from July 2016 to June 2018.

Statistical Analyses

We analyzed three primary outcomes: (1) general CPV-measured improvements in each round (scored against evidence-based scoring criteria); (2) disease-specific CPV improvements over each round; and (3) changes in patient-level outcomes and economic savings among AdventHealth pneumonia/sepsis and heart failure patients from the aforementioned improvements. We used Student’s t-test to analyze continuous outcome variables (including CPV, cost of care, and length of stay data) and Fisher’s exact test for binary outcome data. All statistical analyses were performed using Stata 14.2 (StataCorp LLC, College Station, Texas).

Baseline Characteristics and Assessment

A total of 107 AdventHealth hospitalists participated in this study (Appendix Table 1). 78.1% of these providers rated the organization’s focus on quality and lowering unnecessary costs as either “good” or “excellent,” but 78.8% also reported that variation in care provided by the group was “moderate” to “very high”.

At baseline, we observed high variability in the care of pneumonia patients with sepsis (pneumonia/sepsis) and heart failure patients as measured by the care decisions obtained in the CPV cases. The overall quality score, which is a weighted average across all domains, averaged 61.9% ± 10.5% for the group (Table 1). Disaggregating scores by condition, we found an average overall score of 59.4% ± 10.9% for pneumonia/sepsis and 64.4% ± 9.4% for heart failure. The diagnosis and treatment domains, which require the most clinical judgment, had the lowest average domain scores of 53.4% ± 20.9% and 51.6% ± 15.1%, respectively.

Changes in CPV Scores

To determine the impact of serial measurement and feedback, we compared performance in the first two rounds of the project with the last two rounds. We found that overall CPV quality scores showed a 4.8%-point absolute improvement (P < .001; Table 1). We saw improvements in all care domains, and those increases were significant in all but the workup (P = .470); the most significant increase was in diagnostic accuracy (+19.1%; P < .001).

By condition, scores showed similar, statistically significant overall improvements: +4.4%-points for pneumonia/sepsis (P = .001) and +5.5%-points for heart failure (P < .001) driven by increases in the diagnosis and treatment domains. For example, providers increased appropriate identification of HF severity by 21.5%-points (P < .001) and primary diagnosis of pneumonia/sepsis by 3.6%-points (P = .385).

In the treatment domain, which included clinical decisions related to initial management and follow-up care, there were several specific improvements. For HF, we found that performing all the essential treatment elements—prescribing diuretics, ACE inhibitors and beta blockers for appropriate patients—improved by 13.9%-points (P = .038); ordering VTE prophylaxis increased more than threefold, from 16.6% to 51.0% (P < .001; Table 2). For pneumonia/sepsis patients, absolute adherence to all four elements of the 3-hour sepsis bundle improved by 11.7%-points (P = .034). We also saw a decrease in low-value diagnostic workup items for patient cases in which the guidelines suggest they are not needed, such as urinary antigen testing, which declined by 14.6%-points (P = .001) and sputum cultures, which declined 26.4%-points (P = .004). In addition, outlining an evidence-based discharge plan including a follow-up visit, patient education and medication reconciliation improved, especially for pneumonia/sepsis patients by 24.3%-points (P < .001).

Patient-Level Impact on LOS and Cost Per Case

We took advantage of the quasi-experimental design, in which only a portion of AdventHealth facilities participated in the project, to compare patient-level results from AQQP-participating physicians against the engagement-matched cohort of hospitalists at nonparticipating AdventHealth facilities. We adjusted for potential differences in patient-level case mix between the two groups by comparing the observed/expected (O/E) LOS and cost per case ratios for pneumonia/sepsis and heart failure patients.

At baseline, AQQP-hospitalists performed better on geometric LOS versus the comparator group (O/E of 1.13 vs 1.22; P = .006) but at about the same on cost per case (O/E of 1.16 vs 1.14; P = .390). Throughout the project, as patient volumes and expected per patient costs rose for both groups, O/E ratios improved among both AQQP and non-AQQP providers.

To set apart the contribution of system-wide improvements from the AQQP project-specific impacts, we applied the O/E improvement rates seen in the comparator group to the AQQP group baseline performance. We then compared that to the actual changes seen in the AQQP throughout the project to see if there was any additional benefit from the simulation measurement and feedback (Figure).

Provider Satisfaction

At the project conclusion, we administered a brief survey. Participants were asked to rate aspects of the project (a five-point Likert scale with five being the highest), and 24 responded. The mean ratings of the relevance of the project to their practice and the overall quality of the material were 4.5 and 4.2, respectively. Providers found the individual feedback reports (3.9) slightly more helpful than the webcast group discussions (3.7; Appendix Table 2 ).

As health systems expand, the opportunity to standardize clinical practice within a system has the potential to enhance patient care and lower costs. However, achieving these goals is challenging when providers are dispersed across geographically separated sites and clinical decision-making is difficult to measure in a standardized way.^16,17 We brought together over 100 physicians and APPs from eight different-sized hospitals in five different states to prospectively determine if we could improve care using a standardized measurement and feedback system. At baseline, we found that care varied dramatically among providers. Care varied in terms of diagnostic accuracy and treatment, which directly relate to care quality and outcomes.⁴ After serial measurement and feedback, we saw reductions in unnecessary testing, more guideline-based treatment decisions, and better discharge planning in the clinical vignettes.

We confirmed that changes in CPV-measured practice translated into lower costs and shorter LOS at the patient level. We further validated the improvements through a quasi-experimental design that compared these changes to those at nonparticipating AdventHealth facilities. We saw more significant cost reductions and decreases in LOS in the simulation-based measurement and feedback cohort with the biggest impact early on. The overall savings to the system, attributable specifically to the AQQP approach, is estimated to be $2.4 million.

One advantage of the online case simulation approach is the ability to bring geographically remote sites together in a shared quality-of-care discussion. The interventions specifically sought to remove barriers between facilities. For example, individual feedback reports allowed providers to see how they compare with providers at other AdventHealth facilities and webcast results discussions enable providers across facilities to discuss specific care decisions.

There were several limitations to the study. While the quasi-experimental design allowed us to make informative comparisons between AQQP-participating facilities and nonparticipating facilities, the assignments were not random, and participants were generally from higher performing hospital medicine groups. The determination of secular versus CPV-related improvement is confounded by other system improvement initiatives that may have impacted cost and LOS results. This is mitigated by the observation that facilities that opted to participate performed better at baseline in risk-adjusted LOS but slightly worse in cost per case, indicating that baseline differences were not dramatic. While both groups improved over time, the QURE measurement and feedback approach led to larger and more rapid gains than those seen in the comparator group. However, we could not exclude the potential that project participation at the site level was biased to those groups disposed to performance improvement. In addition, our patient-level data analysis was limited to the metrics available and did not allow us to directly compare patient-level performance across the plethora of clinically relevant CPV data that showed improvement. Our inpatient cost per case analysis showed significant savings for the system but did not include all potentially favorable economic impacts such as lower follow-up care costs for patients, more accurate reimbursement through better coding or fewer lost days of productivity.

With continued consolidation in healthcare and broader health systems spanning multiple geographies, new tools are needed to support standardized, evidence-based care across sites. This standardization is especially important, both clinically and financially, for high-volume, high-cost diseases such as sepsis and heart failure. However, changing practice cannot happen without collaborative engagement with providers. Standardized patient vignettes are an opportunity to measure and provide feedback in a systematic way that engages providers and is particularly well-suited to large systems and common clinical conditions. This analysis, from a real-world study, shows that an approach that standardizes care and lowers costs may be particularly helpful for large systems needing to bring disparate sites together as they concurrently move toward value-based payment.

Disclosures

QURE, LLC, whose intellectual property was used to prepare the cases and collect the data, was contracted by AdventHealth. Otherwise, any of the study authors report no potential conflicts to disclose.

Funding

This work was funded by a contract between AdventHealth (formerly Adventist Health System) and QURE, LLC.