Hospital Medicine

Millennials, defined as those born between 1981 and 1996, currently comprise 15% of all active physicians in the US.^1,2 A recent survey found that nearly 4 of 5 US millennial physicians have a desire for cross-sectional work in areas beyond patient care, such as academic research, health care consulting, entrepreneurship, and health care administration.³

For employers and educators, a better understanding of these preferences, through consideration of the unique education and skill set of the millennial physician workforce, may lead to more effective recruitment of young physicians and improved health systems, avoiding a mismatch between health care provider skills and available jobs that can be costly for both employers and employees.⁴

This article describes how US millennial physicians are choosing to cross-train (obtaining multiple degrees and/or completing combined medical residency training) throughout undergraduate, medical, and graduate medical education. We also outline ways in which the current physician marketplace may not match the skills of this population and suggest some ways that health care organizations could capitalize on this trend toward more cross-trained personnel in order to effectively recruit and retain the next generation of physicians.

Millennial Education

Undergraduates

The number of interdisciplinary undergraduate majors increased by almost 250% from 1975 to 2000.⁵ In 2010, nearly 20% of US college students graduated with 2 majors, representing a 70% increase in double majors between 2001 and 2011.^6,7 One emerging category of interdisciplinary majors in US colleges is health humanities programs, which have quadrupled since 2000.⁸

Medical school applicants and matriculants reflect this trend. Whereas in 1994, only 19% of applicants to medical school held nonscience degrees, about one-third of applicants now hold such degrees.^9,10 We have found no aggregated data on double majors entering US medical schools, but public class profiles suggest that medical school matriculants mirror their undergraduate counterparts in their tendency to hold double majors. In 2016, for example, 15% of the incoming class at the University of Michigan Medical School was composed of double majors, increasing to over 25% in 2017.¹¹

Medical Students

Early dual-degree programs in undergraduate medical training were reserved for MD/PhD programs.¹² Most US MD/PhD programs (90 out of 151) now offer doctorates in social sciences, humanities, or other nontraditional fields of graduate medical study, reflecting a shift in interests of those seeking dual-degree training in undergraduate medical education.¹³ While only 3 MD/PhD programs in the 1970s included trainees in the social sciences, 17 such programs exist today.¹⁴

Interest in dual-degree programs offering master’s level study has also increased over the past decade. In 2017, 87 medical schools offered programs for students to pursue a master of public health (MPH) and 41 offered master of science degrees in various fields, up from 52 and 37 institutions, respectively in 2006.¹⁵ The number of schools offering combined training in nonscience fields has also grown, with 63 institutions now offering a master of business administration (MBA), nearly double the number offered in 2006.¹⁵ At some institutions more than 20% of students are earning a master’s degree or doctorate in addition to their MD degree.¹⁶

Residents

The authors found no documentation of US residency training programs, outside of those in the specialty of preventive medicine, providing trainees with formal opportunities to obtain an MBA or MPH prior to 2001.¹⁷ However, of the 510 internal medicine residency programs listed on the American Medical Association residency and fellowship database (freida.ama-assn.org), 45 identified as having established a pathway for residents to pursue an MBA, MPH, or PhD during residency.¹⁸

Over the past 20 years, combined residency programs have increased 49% (from 128 to 191), which is triple the 16% rate (1,350 to 1,562) of increase in programs in internal medicine, pediatrics, family medicine, psychiatry, and emergency medicine.^19,20 A 2009 moratorium on the creation of new combined residency programs in psychiatry and neurology was lifted in 2016and is likely to increase the rate of total combined programs.²¹

The Table shows the number of categorical and combined residency programs available in 1996 and in 2016. Over 2 decades, 17 new specialty combinations became available for residency training. While there were no combined training programs within these 17 new combinations in 1996,there were 66 programs with these combinations in 2016.^19,20

Although surgical specialties are notably absent from the list of combined residency options, likely due to the duration of surgical training, some surgical training programs do offer pathways that culminate in combined degrees,²² and a high number of surgery program directors agree that residents should receive formal training in business and practice management.²³

The Medical Job Market

Although today’s young physicians are cross-trained in multiple disciplines, the current job market may not directly match these skill sets. Of the 7,235 jobs listed by the New England Journal of Medicine (NEJM) career center (www.nejmcareercenter.org/jobs), only 54 were targeted at those with combined training, the majority of which were aimed at those trained in internal medicine/pediatrics. Of the combined specialties in the Table, formal positions were listed for only 6.²⁴ A search of nearly 1,500 federal medical positions on USAJOBS (www.usajobs.gov) found only 4 jobs that combined specialties, all restricted to internal medicine/pediatrics.²⁵ When searching for jobs containing the terms MBA, MPH, and public health there were only 8 such positions on NEJM and 7 on USAJOBS.^24,25 Although the totality of the medical marketplace may not be best encompassed by these sources, the authors believe NEJM and USAJOBS are somewhat representative of the opportunities for physicians in the US.

Medical jobs tailored to cross-trained physicians do not appear to have kept pace with the numbers of such specialists currently in medical school and residency training. Though millennials are cross-training in increasing numbers, we surmise that they are not doing so as a direct result of the job market.

Future Medicine

Regardless of the mismatch between cross-trained physicians and the current job market, millennials may be well suited for future health systems. In 2001, the National Academies of Sciences, Engineering and Medicine (NASEM) called for increasing interdisciplinary training and improving cross-functional team performance as a major goal for health care providers in twenty-first century health systems.²⁶ NASEM also recommended that academic medical centers develop medical leaders who can manage systems changes required to enhance health, a proposal supported by the fact that hospitals with medically trained CEOs outperform others.^27,28

Public Health 3.0, a federal initiative to improve and integrate public health efforts, also emphasizes cross-disciplinary teams and cross-sector partnerships,²⁹ while the Centers for Medicare and Medicaid Services (CMS) has incentivized the development of interprofessional health care teams.³⁰ While cross-training does not automatically connote interdisciplinary training, we believe that cross-training may reveal or develop an interdisciplinary mind-set that may support and embrace interdisciplinary performance. Finally, the US Department of Health and Human Services’ (HHS) Strategic Goals emphasize integrated care for vulnerable populations, something that cross-trained physicians may be especially poised to accomplish.³¹

A Path Forward

The education, training, and priorities of young physicians demonstrates career interests that diverge from mainstream, traditional options. Data provided herein describe the increasing rates at which millennial physicians are cross-training and have suggested that the current marketplace may not match the interests of this population. The ultimate question is where such cross-trained physicians fit into today’s (or tomorrow’s) health system?

It may be easiest to deploy cross-trained physicians in their respective clinical departments (eg, having a physician trained in internal medicine and pediatrics perform clinical duties in both a medicine department and a pediatrics department). But < 40% of dual-boarded physicians practice both specialties in which they’re trained, so other opportunities should be pursued.^32,33 One strategy may be to embrace the promise of interdisciplinary care, as supported by Public Health 3.0 and NASEM.^26,29 Our evidence may demonstrate that the interdisciplinary mind-set may be more readily evident in the millennial generation, and that this mind-set may improve interdisciplinary care.

As health is impacted both by direct clinical care as well as programs designed to address population health, cross-trained physicians may be better equipped to integrate aspects of clinical care spanning a variety of clinical fields as well as orchestrating programs designed to improve health at the population level. This mind-set may be best captured by organizations willing to adapt their medical positions to emphasize multidisciplinary training, skills, and capabilities. For example, a physician trained in internal medicine and psychiatry may have the unique training and skill-set to establish an integrated behavioral health clinic that crosses boundaries between traditional departments, emphasizing the whole health of the clinic’s population and not simply focusing on providing services of a particular specialty. Hiring cross-trained physicians throughout such a clinic may benefit the operations of the clinic and improve not only the services provided, but ultimately, the health of that clinic’s patients. By embracing cross-trained physicians, health care organizations and educators may better meet the needs of their employees, likely resulting in a more cost-effective investment for employers, employees, and the health system as a whole.⁴ Additionally, patient health may also improve.

There is evidence that cross-trained physicians are already likely to hold leadership positions compared with their categorically-trained counterparts, and this may reflect the benefits of an interdisciplinary mind-set.³³ Perhaps a cross-trained physician is more likely to see beyond standard, specialty-based institutional barriers and develop processes and programs designed for overall patient benefit. Leadership is a skill that many millennials clearly wish to enhance throughout their career.³⁴ Recruiting cross-trained physicians for leadership positions may reveal synergies between such training and an ability to lead health care organizations into the future.

Many millennial physicians are bringing a new set of skills into the medical marketplace. Health organizations should identify ways to recruit for these skills and deploy them within their systems in order to have more dedicated, engaged employees, more effective health systems, and ultimately, healthier patients.

Acknowledgments
Data from this analysis were presented at the 10th Consortium of Universities for Global Health conference in 2019.³⁵

Millennials, defined as those born between 1981 and 1996, currently comprise 15% of all active physicians in the US.^1,2 A recent survey found that nearly 4 of 5 US millennial physicians have a desire for cross-sectional work in areas beyond patient care, such as academic research, health care consulting, entrepreneurship, and health care administration.³

For employers and educators, a better understanding of these preferences, through consideration of the unique education and skill set of the millennial physician workforce, may lead to more effective recruitment of young physicians and improved health systems, avoiding a mismatch between health care provider skills and available jobs that can be costly for both employers and employees.⁴

This article describes how US millennial physicians are choosing to cross-train (obtaining multiple degrees and/or completing combined medical residency training) throughout undergraduate, medical, and graduate medical education. We also outline ways in which the current physician marketplace may not match the skills of this population and suggest some ways that health care organizations could capitalize on this trend toward more cross-trained personnel in order to effectively recruit and retain the next generation of physicians.

Millennial Education

Undergraduates

The number of interdisciplinary undergraduate majors increased by almost 250% from 1975 to 2000.⁵ In 2010, nearly 20% of US college students graduated with 2 majors, representing a 70% increase in double majors between 2001 and 2011.^6,7 One emerging category of interdisciplinary majors in US colleges is health humanities programs, which have quadrupled since 2000.⁸

Medical school applicants and matriculants reflect this trend. Whereas in 1994, only 19% of applicants to medical school held nonscience degrees, about one-third of applicants now hold such degrees.^9,10 We have found no aggregated data on double majors entering US medical schools, but public class profiles suggest that medical school matriculants mirror their undergraduate counterparts in their tendency to hold double majors. In 2016, for example, 15% of the incoming class at the University of Michigan Medical School was composed of double majors, increasing to over 25% in 2017.¹¹

Medical Students

Early dual-degree programs in undergraduate medical training were reserved for MD/PhD programs.¹² Most US MD/PhD programs (90 out of 151) now offer doctorates in social sciences, humanities, or other nontraditional fields of graduate medical study, reflecting a shift in interests of those seeking dual-degree training in undergraduate medical education.¹³ While only 3 MD/PhD programs in the 1970s included trainees in the social sciences, 17 such programs exist today.¹⁴

Interest in dual-degree programs offering master’s level study has also increased over the past decade. In 2017, 87 medical schools offered programs for students to pursue a master of public health (MPH) and 41 offered master of science degrees in various fields, up from 52 and 37 institutions, respectively in 2006.¹⁵ The number of schools offering combined training in nonscience fields has also grown, with 63 institutions now offering a master of business administration (MBA), nearly double the number offered in 2006.¹⁵ At some institutions more than 20% of students are earning a master’s degree or doctorate in addition to their MD degree.¹⁶

Residents

The authors found no documentation of US residency training programs, outside of those in the specialty of preventive medicine, providing trainees with formal opportunities to obtain an MBA or MPH prior to 2001.¹⁷ However, of the 510 internal medicine residency programs listed on the American Medical Association residency and fellowship database (freida.ama-assn.org), 45 identified as having established a pathway for residents to pursue an MBA, MPH, or PhD during residency.¹⁸

Over the past 20 years, combined residency programs have increased 49% (from 128 to 191), which is triple the 16% rate (1,350 to 1,562) of increase in programs in internal medicine, pediatrics, family medicine, psychiatry, and emergency medicine.^19,20 A 2009 moratorium on the creation of new combined residency programs in psychiatry and neurology was lifted in 2016and is likely to increase the rate of total combined programs.²¹

The Table shows the number of categorical and combined residency programs available in 1996 and in 2016. Over 2 decades, 17 new specialty combinations became available for residency training. While there were no combined training programs within these 17 new combinations in 1996,there were 66 programs with these combinations in 2016.^19,20

Although surgical specialties are notably absent from the list of combined residency options, likely due to the duration of surgical training, some surgical training programs do offer pathways that culminate in combined degrees,²² and a high number of surgery program directors agree that residents should receive formal training in business and practice management.²³

The Medical Job Market

Although today’s young physicians are cross-trained in multiple disciplines, the current job market may not directly match these skill sets. Of the 7,235 jobs listed by the New England Journal of Medicine (NEJM) career center (www.nejmcareercenter.org/jobs), only 54 were targeted at those with combined training, the majority of which were aimed at those trained in internal medicine/pediatrics. Of the combined specialties in the Table, formal positions were listed for only 6.²⁴ A search of nearly 1,500 federal medical positions on USAJOBS (www.usajobs.gov) found only 4 jobs that combined specialties, all restricted to internal medicine/pediatrics.²⁵ When searching for jobs containing the terms MBA, MPH, and public health there were only 8 such positions on NEJM and 7 on USAJOBS.^24,25 Although the totality of the medical marketplace may not be best encompassed by these sources, the authors believe NEJM and USAJOBS are somewhat representative of the opportunities for physicians in the US.

Medical jobs tailored to cross-trained physicians do not appear to have kept pace with the numbers of such specialists currently in medical school and residency training. Though millennials are cross-training in increasing numbers, we surmise that they are not doing so as a direct result of the job market.

Future Medicine

Regardless of the mismatch between cross-trained physicians and the current job market, millennials may be well suited for future health systems. In 2001, the National Academies of Sciences, Engineering and Medicine (NASEM) called for increasing interdisciplinary training and improving cross-functional team performance as a major goal for health care providers in twenty-first century health systems.²⁶ NASEM also recommended that academic medical centers develop medical leaders who can manage systems changes required to enhance health, a proposal supported by the fact that hospitals with medically trained CEOs outperform others.^27,28

Public Health 3.0, a federal initiative to improve and integrate public health efforts, also emphasizes cross-disciplinary teams and cross-sector partnerships,²⁹ while the Centers for Medicare and Medicaid Services (CMS) has incentivized the development of interprofessional health care teams.³⁰ While cross-training does not automatically connote interdisciplinary training, we believe that cross-training may reveal or develop an interdisciplinary mind-set that may support and embrace interdisciplinary performance. Finally, the US Department of Health and Human Services’ (HHS) Strategic Goals emphasize integrated care for vulnerable populations, something that cross-trained physicians may be especially poised to accomplish.³¹

A Path Forward

The education, training, and priorities of young physicians demonstrates career interests that diverge from mainstream, traditional options. Data provided herein describe the increasing rates at which millennial physicians are cross-training and have suggested that the current marketplace may not match the interests of this population. The ultimate question is where such cross-trained physicians fit into today’s (or tomorrow’s) health system?

It may be easiest to deploy cross-trained physicians in their respective clinical departments (eg, having a physician trained in internal medicine and pediatrics perform clinical duties in both a medicine department and a pediatrics department). But < 40% of dual-boarded physicians practice both specialties in which they’re trained, so other opportunities should be pursued.^32,33 One strategy may be to embrace the promise of interdisciplinary care, as supported by Public Health 3.0 and NASEM.^26,29 Our evidence may demonstrate that the interdisciplinary mind-set may be more readily evident in the millennial generation, and that this mind-set may improve interdisciplinary care.

As health is impacted both by direct clinical care as well as programs designed to address population health, cross-trained physicians may be better equipped to integrate aspects of clinical care spanning a variety of clinical fields as well as orchestrating programs designed to improve health at the population level. This mind-set may be best captured by organizations willing to adapt their medical positions to emphasize multidisciplinary training, skills, and capabilities. For example, a physician trained in internal medicine and psychiatry may have the unique training and skill-set to establish an integrated behavioral health clinic that crosses boundaries between traditional departments, emphasizing the whole health of the clinic’s population and not simply focusing on providing services of a particular specialty. Hiring cross-trained physicians throughout such a clinic may benefit the operations of the clinic and improve not only the services provided, but ultimately, the health of that clinic’s patients. By embracing cross-trained physicians, health care organizations and educators may better meet the needs of their employees, likely resulting in a more cost-effective investment for employers, employees, and the health system as a whole.⁴ Additionally, patient health may also improve.

There is evidence that cross-trained physicians are already likely to hold leadership positions compared with their categorically-trained counterparts, and this may reflect the benefits of an interdisciplinary mind-set.³³ Perhaps a cross-trained physician is more likely to see beyond standard, specialty-based institutional barriers and develop processes and programs designed for overall patient benefit. Leadership is a skill that many millennials clearly wish to enhance throughout their career.³⁴ Recruiting cross-trained physicians for leadership positions may reveal synergies between such training and an ability to lead health care organizations into the future.

Many millennial physicians are bringing a new set of skills into the medical marketplace. Health organizations should identify ways to recruit for these skills and deploy them within their systems in order to have more dedicated, engaged employees, more effective health systems, and ultimately, healthier patients.

Acknowledgments
Data from this analysis were presented at the 10th Consortium of Universities for Global Health conference in 2019.³⁵

Millennials, defined as those born between 1981 and 1996, currently comprise 15% of all active physicians in the US.^1,2 A recent survey found that nearly 4 of 5 US millennial physicians have a desire for cross-sectional work in areas beyond patient care, such as academic research, health care consulting, entrepreneurship, and health care administration.³

For employers and educators, a better understanding of these preferences, through consideration of the unique education and skill set of the millennial physician workforce, may lead to more effective recruitment of young physicians and improved health systems, avoiding a mismatch between health care provider skills and available jobs that can be costly for both employers and employees.⁴

This article describes how US millennial physicians are choosing to cross-train (obtaining multiple degrees and/or completing combined medical residency training) throughout undergraduate, medical, and graduate medical education. We also outline ways in which the current physician marketplace may not match the skills of this population and suggest some ways that health care organizations could capitalize on this trend toward more cross-trained personnel in order to effectively recruit and retain the next generation of physicians.

Millennial Education

Undergraduates

The number of interdisciplinary undergraduate majors increased by almost 250% from 1975 to 2000.⁵ In 2010, nearly 20% of US college students graduated with 2 majors, representing a 70% increase in double majors between 2001 and 2011.^6,7 One emerging category of interdisciplinary majors in US colleges is health humanities programs, which have quadrupled since 2000.⁸

Medical school applicants and matriculants reflect this trend. Whereas in 1994, only 19% of applicants to medical school held nonscience degrees, about one-third of applicants now hold such degrees.^9,10 We have found no aggregated data on double majors entering US medical schools, but public class profiles suggest that medical school matriculants mirror their undergraduate counterparts in their tendency to hold double majors. In 2016, for example, 15% of the incoming class at the University of Michigan Medical School was composed of double majors, increasing to over 25% in 2017.¹¹

Medical Students

Early dual-degree programs in undergraduate medical training were reserved for MD/PhD programs.¹² Most US MD/PhD programs (90 out of 151) now offer doctorates in social sciences, humanities, or other nontraditional fields of graduate medical study, reflecting a shift in interests of those seeking dual-degree training in undergraduate medical education.¹³ While only 3 MD/PhD programs in the 1970s included trainees in the social sciences, 17 such programs exist today.¹⁴

Interest in dual-degree programs offering master’s level study has also increased over the past decade. In 2017, 87 medical schools offered programs for students to pursue a master of public health (MPH) and 41 offered master of science degrees in various fields, up from 52 and 37 institutions, respectively in 2006.¹⁵ The number of schools offering combined training in nonscience fields has also grown, with 63 institutions now offering a master of business administration (MBA), nearly double the number offered in 2006.¹⁵ At some institutions more than 20% of students are earning a master’s degree or doctorate in addition to their MD degree.¹⁶

Residents

The authors found no documentation of US residency training programs, outside of those in the specialty of preventive medicine, providing trainees with formal opportunities to obtain an MBA or MPH prior to 2001.¹⁷ However, of the 510 internal medicine residency programs listed on the American Medical Association residency and fellowship database (freida.ama-assn.org), 45 identified as having established a pathway for residents to pursue an MBA, MPH, or PhD during residency.¹⁸

Over the past 20 years, combined residency programs have increased 49% (from 128 to 191), which is triple the 16% rate (1,350 to 1,562) of increase in programs in internal medicine, pediatrics, family medicine, psychiatry, and emergency medicine.^19,20 A 2009 moratorium on the creation of new combined residency programs in psychiatry and neurology was lifted in 2016and is likely to increase the rate of total combined programs.²¹

The Table shows the number of categorical and combined residency programs available in 1996 and in 2016. Over 2 decades, 17 new specialty combinations became available for residency training. While there were no combined training programs within these 17 new combinations in 1996,there were 66 programs with these combinations in 2016.^19,20

Although surgical specialties are notably absent from the list of combined residency options, likely due to the duration of surgical training, some surgical training programs do offer pathways that culminate in combined degrees,²² and a high number of surgery program directors agree that residents should receive formal training in business and practice management.²³

The Medical Job Market

Although today’s young physicians are cross-trained in multiple disciplines, the current job market may not directly match these skill sets. Of the 7,235 jobs listed by the New England Journal of Medicine (NEJM) career center (www.nejmcareercenter.org/jobs), only 54 were targeted at those with combined training, the majority of which were aimed at those trained in internal medicine/pediatrics. Of the combined specialties in the Table, formal positions were listed for only 6.²⁴ A search of nearly 1,500 federal medical positions on USAJOBS (www.usajobs.gov) found only 4 jobs that combined specialties, all restricted to internal medicine/pediatrics.²⁵ When searching for jobs containing the terms MBA, MPH, and public health there were only 8 such positions on NEJM and 7 on USAJOBS.^24,25 Although the totality of the medical marketplace may not be best encompassed by these sources, the authors believe NEJM and USAJOBS are somewhat representative of the opportunities for physicians in the US.

Medical jobs tailored to cross-trained physicians do not appear to have kept pace with the numbers of such specialists currently in medical school and residency training. Though millennials are cross-training in increasing numbers, we surmise that they are not doing so as a direct result of the job market.

Future Medicine

Regardless of the mismatch between cross-trained physicians and the current job market, millennials may be well suited for future health systems. In 2001, the National Academies of Sciences, Engineering and Medicine (NASEM) called for increasing interdisciplinary training and improving cross-functional team performance as a major goal for health care providers in twenty-first century health systems.²⁶ NASEM also recommended that academic medical centers develop medical leaders who can manage systems changes required to enhance health, a proposal supported by the fact that hospitals with medically trained CEOs outperform others.^27,28

Public Health 3.0, a federal initiative to improve and integrate public health efforts, also emphasizes cross-disciplinary teams and cross-sector partnerships,²⁹ while the Centers for Medicare and Medicaid Services (CMS) has incentivized the development of interprofessional health care teams.³⁰ While cross-training does not automatically connote interdisciplinary training, we believe that cross-training may reveal or develop an interdisciplinary mind-set that may support and embrace interdisciplinary performance. Finally, the US Department of Health and Human Services’ (HHS) Strategic Goals emphasize integrated care for vulnerable populations, something that cross-trained physicians may be especially poised to accomplish.³¹

A Path Forward

The education, training, and priorities of young physicians demonstrates career interests that diverge from mainstream, traditional options. Data provided herein describe the increasing rates at which millennial physicians are cross-training and have suggested that the current marketplace may not match the interests of this population. The ultimate question is where such cross-trained physicians fit into today’s (or tomorrow’s) health system?

It may be easiest to deploy cross-trained physicians in their respective clinical departments (eg, having a physician trained in internal medicine and pediatrics perform clinical duties in both a medicine department and a pediatrics department). But < 40% of dual-boarded physicians practice both specialties in which they’re trained, so other opportunities should be pursued.^32,33 One strategy may be to embrace the promise of interdisciplinary care, as supported by Public Health 3.0 and NASEM.^26,29 Our evidence may demonstrate that the interdisciplinary mind-set may be more readily evident in the millennial generation, and that this mind-set may improve interdisciplinary care.

As health is impacted both by direct clinical care as well as programs designed to address population health, cross-trained physicians may be better equipped to integrate aspects of clinical care spanning a variety of clinical fields as well as orchestrating programs designed to improve health at the population level. This mind-set may be best captured by organizations willing to adapt their medical positions to emphasize multidisciplinary training, skills, and capabilities. For example, a physician trained in internal medicine and psychiatry may have the unique training and skill-set to establish an integrated behavioral health clinic that crosses boundaries between traditional departments, emphasizing the whole health of the clinic’s population and not simply focusing on providing services of a particular specialty. Hiring cross-trained physicians throughout such a clinic may benefit the operations of the clinic and improve not only the services provided, but ultimately, the health of that clinic’s patients. By embracing cross-trained physicians, health care organizations and educators may better meet the needs of their employees, likely resulting in a more cost-effective investment for employers, employees, and the health system as a whole.⁴ Additionally, patient health may also improve.

There is evidence that cross-trained physicians are already likely to hold leadership positions compared with their categorically-trained counterparts, and this may reflect the benefits of an interdisciplinary mind-set.³³ Perhaps a cross-trained physician is more likely to see beyond standard, specialty-based institutional barriers and develop processes and programs designed for overall patient benefit. Leadership is a skill that many millennials clearly wish to enhance throughout their career.³⁴ Recruiting cross-trained physicians for leadership positions may reveal synergies between such training and an ability to lead health care organizations into the future.

Many millennial physicians are bringing a new set of skills into the medical marketplace. Health organizations should identify ways to recruit for these skills and deploy them within their systems in order to have more dedicated, engaged employees, more effective health systems, and ultimately, healthier patients.

Acknowledgments
Data from this analysis were presented at the 10th Consortium of Universities for Global Health conference in 2019.³⁵

Readers may recall that at the end of each calendar as opposed to fiscal year—I know it is hard to believe time exists outside the Federal system—Federal Practitioner publishes my ethics-focused version of the familiar year-end roundup. This year I am reversing the typical order of most annual rankings by putting the worst first for 2 morally salient reasons.

The first is that, sadly, it is almost always easier to identify multiple incidents that compete ignominiously for the “worst” of federal health care. Even more disappointing, it is comparatively difficult to find stories for the “best” that are of the same scale and scope as the bad news. This is not to say that every day there are not individual narratives of courage and compassion reported in US Department of Defense, US Public Health Service, and US Department of Veterans Affairs (VA), and hundreds more unsung heroes.

The second reason is that as human beings our psychology is such that we gravitate toward the worst things more powerfully and persistently than we do the best. This is in part why it is more difficult to find uplifting stories and why the demoralizing ones affect us so strongly. In an exhaustive review of the subject, psychologists Roy Baumeister and colleagues conclude that,

When equal measures of good and bad are present, however, the psychological effects of bad ones outweigh those of the good ones. This may in fact be a general principle or law of psychological phenomena, possibly reflecting the innate predispositions of the psyche or at least reflecting the almost inevitable adaptation of each individual to the exigencies of daily life.²

I am thus saving the best for last in the hope that it will be more memorable and impactful than the worst.

Unique to this year’s look-back, both the negative and the positive accounts come from the domain of end-of-life care. And unlike prior reviews where the lack of administrative vigilance and professional competence affected hundreds of patients, families, and staff, each of this year’s incidents involve a single patient.

An incident that occurred in September 2019 at a VA Community Living Center (CLC) in Georgia stood out in infamy apart from all others. It was the report of a veteran in a VA nursing home who had been bitten more than 100 times by ants crawling all over his room. He died shortly afterward. In a scene out of a horror movie tapping into the most primeval human fears, his daughter Laquana Ross described her father, a Vietnam Air Force veteran with cancer, to media and VA officials in graphic terms. “I understand mistakes happen,” she said. “I’ve had ants. But he was bit by ants two days in a row. They feasted on him.”³

In this new era of holding its senior executive service accountable, the outraged chair of the Senate Veterans Affairs Committee demanded that heads roll, and the VA acted rapidly to comply.⁴ The VA Central Office placed the network director on administrative leave, reassigned the chief medical officer, and initiated quality and safety reviews as well as an administrative investigative board to scrutinize how the parent Atlanta VA medical center managed the situation. In total, 9 officials connected to the incident were placed on leave. The VA apologized, with VA Secretary Robert Wilke zeroing in on the core values involved in the tragedy, “This is about basic humanity and dignity,” he said. “I don’t care what steps were taken to address the issues. We did not treat a vet with the dignity that he and his family deserved.”⁵ Yet it was the veteran’s daughter, with unbelievable charity, who asked the most crucial question that must be answered within the framework of a just culture if similar tragedies are not to occur in the future, “I know the staff, without a shadow of doubt, respected my dad and even loved him,” Ross said. “But what’s their ability to assess situations and fix things?”³

To begin to give Ms. Ross the answer she deserves, we must understand that the antithesis of love is not hate but indifference; of compassion, it is not cruelty but coldness. A true just culture reserves individual blame for those who have ill-will and adopts a systems perspective of organizational improvement toward most other types of errors.⁶ This means that the deplorable conditions in the CLC cannot be charged to the failure of a single staff member to fulfil their obligations but to collective collapse at many levels of the organization. Just culture is ethically laudable and far superior to the history in federal service of capricious punishment or institutional apathy that far too often were the default reactions to media exposures or congressional ire. Justice, though necessary, is not sufficient to achieve virtue. Those who work in health care also must be inspired to offer mercy, kindness, and compassion, especially in our most sacred privilege to provide care of the dying.

The best of 2019 illustrates this distinction movingly. This account also involves a Vietnam veteran, this time a Marine also dying of cancer, which happened just about a month after the earlier report. To be transparent it occurred at my home VA medical center in New Mexico. I was peripherally involved in the case as a consultant but had no role in the wondrous things that transpired. The last wish of a patient dying in the hospice unit on campus was to see his beloved dog who had been taken to the local city animal shelter when he was hospitalized because he had no friends or family to look after the companion animal. A social worker on the palliative care team called the animal shelter and explained the patient did not have much time left but wanted to see his dog before he died. Working together with support from facility leadership, shelter workers brought the dog to visit with the patient for an entire day while hospice staff cried with joy and sadness.⁷

As the epigraph for this editorial from Dame Cicely Saunders, the founder of the modern hospice movement, says, the difference between unspeakable pain and meaningful suffering can be measured in the depth of compassion caregivers show to the dying. It is this quality of mercy that in one case condemns, and in the other praises, us all as health care and administrative professionals in the service of our country. Baumeister and colleagues suggest that the human tendency to magnify the bad and minimize the good in everyday myopia may in a wider vision actually be a reason for hope:

It may be that humans and animals show heightened awareness of and responded more quickly to negative information because it signals a need for change. Hence, the adaptiveness of self-regulation partly lies in the organism’s ability to detect when response modifications are necessary and when they are unnecessary. Moreover, the lessons learned from bad events should ideally be retained permanently so that the same dangers or costs are not encountered repeatedly. Meanwhile, good events (such as those that provide a feeling of satisfaction and contentment) should ideally wear off so that the organism is motivated to continue searching for more and better outcomes.²

Let us all take this lesson into our work in 2020 so that when it comes time to write this column next year in the chilling cold of late autumn there will be more stories of light than darkness from which to choose.

Readers may recall that at the end of each calendar as opposed to fiscal year—I know it is hard to believe time exists outside the Federal system—Federal Practitioner publishes my ethics-focused version of the familiar year-end roundup. This year I am reversing the typical order of most annual rankings by putting the worst first for 2 morally salient reasons.

The first is that, sadly, it is almost always easier to identify multiple incidents that compete ignominiously for the “worst” of federal health care. Even more disappointing, it is comparatively difficult to find stories for the “best” that are of the same scale and scope as the bad news. This is not to say that every day there are not individual narratives of courage and compassion reported in US Department of Defense, US Public Health Service, and US Department of Veterans Affairs (VA), and hundreds more unsung heroes.

The second reason is that as human beings our psychology is such that we gravitate toward the worst things more powerfully and persistently than we do the best. This is in part why it is more difficult to find uplifting stories and why the demoralizing ones affect us so strongly. In an exhaustive review of the subject, psychologists Roy Baumeister and colleagues conclude that,

When equal measures of good and bad are present, however, the psychological effects of bad ones outweigh those of the good ones. This may in fact be a general principle or law of psychological phenomena, possibly reflecting the innate predispositions of the psyche or at least reflecting the almost inevitable adaptation of each individual to the exigencies of daily life.²

I am thus saving the best for last in the hope that it will be more memorable and impactful than the worst.

Unique to this year’s look-back, both the negative and the positive accounts come from the domain of end-of-life care. And unlike prior reviews where the lack of administrative vigilance and professional competence affected hundreds of patients, families, and staff, each of this year’s incidents involve a single patient.

An incident that occurred in September 2019 at a VA Community Living Center (CLC) in Georgia stood out in infamy apart from all others. It was the report of a veteran in a VA nursing home who had been bitten more than 100 times by ants crawling all over his room. He died shortly afterward. In a scene out of a horror movie tapping into the most primeval human fears, his daughter Laquana Ross described her father, a Vietnam Air Force veteran with cancer, to media and VA officials in graphic terms. “I understand mistakes happen,” she said. “I’ve had ants. But he was bit by ants two days in a row. They feasted on him.”³

In this new era of holding its senior executive service accountable, the outraged chair of the Senate Veterans Affairs Committee demanded that heads roll, and the VA acted rapidly to comply.⁴ The VA Central Office placed the network director on administrative leave, reassigned the chief medical officer, and initiated quality and safety reviews as well as an administrative investigative board to scrutinize how the parent Atlanta VA medical center managed the situation. In total, 9 officials connected to the incident were placed on leave. The VA apologized, with VA Secretary Robert Wilke zeroing in on the core values involved in the tragedy, “This is about basic humanity and dignity,” he said. “I don’t care what steps were taken to address the issues. We did not treat a vet with the dignity that he and his family deserved.”⁵ Yet it was the veteran’s daughter, with unbelievable charity, who asked the most crucial question that must be answered within the framework of a just culture if similar tragedies are not to occur in the future, “I know the staff, without a shadow of doubt, respected my dad and even loved him,” Ross said. “But what’s their ability to assess situations and fix things?”³

To begin to give Ms. Ross the answer she deserves, we must understand that the antithesis of love is not hate but indifference; of compassion, it is not cruelty but coldness. A true just culture reserves individual blame for those who have ill-will and adopts a systems perspective of organizational improvement toward most other types of errors.⁶ This means that the deplorable conditions in the CLC cannot be charged to the failure of a single staff member to fulfil their obligations but to collective collapse at many levels of the organization. Just culture is ethically laudable and far superior to the history in federal service of capricious punishment or institutional apathy that far too often were the default reactions to media exposures or congressional ire. Justice, though necessary, is not sufficient to achieve virtue. Those who work in health care also must be inspired to offer mercy, kindness, and compassion, especially in our most sacred privilege to provide care of the dying.

The best of 2019 illustrates this distinction movingly. This account also involves a Vietnam veteran, this time a Marine also dying of cancer, which happened just about a month after the earlier report. To be transparent it occurred at my home VA medical center in New Mexico. I was peripherally involved in the case as a consultant but had no role in the wondrous things that transpired. The last wish of a patient dying in the hospice unit on campus was to see his beloved dog who had been taken to the local city animal shelter when he was hospitalized because he had no friends or family to look after the companion animal. A social worker on the palliative care team called the animal shelter and explained the patient did not have much time left but wanted to see his dog before he died. Working together with support from facility leadership, shelter workers brought the dog to visit with the patient for an entire day while hospice staff cried with joy and sadness.⁷

As the epigraph for this editorial from Dame Cicely Saunders, the founder of the modern hospice movement, says, the difference between unspeakable pain and meaningful suffering can be measured in the depth of compassion caregivers show to the dying. It is this quality of mercy that in one case condemns, and in the other praises, us all as health care and administrative professionals in the service of our country. Baumeister and colleagues suggest that the human tendency to magnify the bad and minimize the good in everyday myopia may in a wider vision actually be a reason for hope:

It may be that humans and animals show heightened awareness of and responded more quickly to negative information because it signals a need for change. Hence, the adaptiveness of self-regulation partly lies in the organism’s ability to detect when response modifications are necessary and when they are unnecessary. Moreover, the lessons learned from bad events should ideally be retained permanently so that the same dangers or costs are not encountered repeatedly. Meanwhile, good events (such as those that provide a feeling of satisfaction and contentment) should ideally wear off so that the organism is motivated to continue searching for more and better outcomes.²

Let us all take this lesson into our work in 2020 so that when it comes time to write this column next year in the chilling cold of late autumn there will be more stories of light than darkness from which to choose.

Readers may recall that at the end of each calendar as opposed to fiscal year—I know it is hard to believe time exists outside the Federal system—Federal Practitioner publishes my ethics-focused version of the familiar year-end roundup. This year I am reversing the typical order of most annual rankings by putting the worst first for 2 morally salient reasons.

The first is that, sadly, it is almost always easier to identify multiple incidents that compete ignominiously for the “worst” of federal health care. Even more disappointing, it is comparatively difficult to find stories for the “best” that are of the same scale and scope as the bad news. This is not to say that every day there are not individual narratives of courage and compassion reported in US Department of Defense, US Public Health Service, and US Department of Veterans Affairs (VA), and hundreds more unsung heroes.

The second reason is that as human beings our psychology is such that we gravitate toward the worst things more powerfully and persistently than we do the best. This is in part why it is more difficult to find uplifting stories and why the demoralizing ones affect us so strongly. In an exhaustive review of the subject, psychologists Roy Baumeister and colleagues conclude that,

When equal measures of good and bad are present, however, the psychological effects of bad ones outweigh those of the good ones. This may in fact be a general principle or law of psychological phenomena, possibly reflecting the innate predispositions of the psyche or at least reflecting the almost inevitable adaptation of each individual to the exigencies of daily life.²

I am thus saving the best for last in the hope that it will be more memorable and impactful than the worst.

Unique to this year’s look-back, both the negative and the positive accounts come from the domain of end-of-life care. And unlike prior reviews where the lack of administrative vigilance and professional competence affected hundreds of patients, families, and staff, each of this year’s incidents involve a single patient.

An incident that occurred in September 2019 at a VA Community Living Center (CLC) in Georgia stood out in infamy apart from all others. It was the report of a veteran in a VA nursing home who had been bitten more than 100 times by ants crawling all over his room. He died shortly afterward. In a scene out of a horror movie tapping into the most primeval human fears, his daughter Laquana Ross described her father, a Vietnam Air Force veteran with cancer, to media and VA officials in graphic terms. “I understand mistakes happen,” she said. “I’ve had ants. But he was bit by ants two days in a row. They feasted on him.”³

In this new era of holding its senior executive service accountable, the outraged chair of the Senate Veterans Affairs Committee demanded that heads roll, and the VA acted rapidly to comply.⁴ The VA Central Office placed the network director on administrative leave, reassigned the chief medical officer, and initiated quality and safety reviews as well as an administrative investigative board to scrutinize how the parent Atlanta VA medical center managed the situation. In total, 9 officials connected to the incident were placed on leave. The VA apologized, with VA Secretary Robert Wilke zeroing in on the core values involved in the tragedy, “This is about basic humanity and dignity,” he said. “I don’t care what steps were taken to address the issues. We did not treat a vet with the dignity that he and his family deserved.”⁵ Yet it was the veteran’s daughter, with unbelievable charity, who asked the most crucial question that must be answered within the framework of a just culture if similar tragedies are not to occur in the future, “I know the staff, without a shadow of doubt, respected my dad and even loved him,” Ross said. “But what’s their ability to assess situations and fix things?”³

To begin to give Ms. Ross the answer she deserves, we must understand that the antithesis of love is not hate but indifference; of compassion, it is not cruelty but coldness. A true just culture reserves individual blame for those who have ill-will and adopts a systems perspective of organizational improvement toward most other types of errors.⁶ This means that the deplorable conditions in the CLC cannot be charged to the failure of a single staff member to fulfil their obligations but to collective collapse at many levels of the organization. Just culture is ethically laudable and far superior to the history in federal service of capricious punishment or institutional apathy that far too often were the default reactions to media exposures or congressional ire. Justice, though necessary, is not sufficient to achieve virtue. Those who work in health care also must be inspired to offer mercy, kindness, and compassion, especially in our most sacred privilege to provide care of the dying.

The best of 2019 illustrates this distinction movingly. This account also involves a Vietnam veteran, this time a Marine also dying of cancer, which happened just about a month after the earlier report. To be transparent it occurred at my home VA medical center in New Mexico. I was peripherally involved in the case as a consultant but had no role in the wondrous things that transpired. The last wish of a patient dying in the hospice unit on campus was to see his beloved dog who had been taken to the local city animal shelter when he was hospitalized because he had no friends or family to look after the companion animal. A social worker on the palliative care team called the animal shelter and explained the patient did not have much time left but wanted to see his dog before he died. Working together with support from facility leadership, shelter workers brought the dog to visit with the patient for an entire day while hospice staff cried with joy and sadness.⁷

As the epigraph for this editorial from Dame Cicely Saunders, the founder of the modern hospice movement, says, the difference between unspeakable pain and meaningful suffering can be measured in the depth of compassion caregivers show to the dying. It is this quality of mercy that in one case condemns, and in the other praises, us all as health care and administrative professionals in the service of our country. Baumeister and colleagues suggest that the human tendency to magnify the bad and minimize the good in everyday myopia may in a wider vision actually be a reason for hope:

It may be that humans and animals show heightened awareness of and responded more quickly to negative information because it signals a need for change. Hence, the adaptiveness of self-regulation partly lies in the organism’s ability to detect when response modifications are necessary and when they are unnecessary. Moreover, the lessons learned from bad events should ideally be retained permanently so that the same dangers or costs are not encountered repeatedly. Meanwhile, good events (such as those that provide a feeling of satisfaction and contentment) should ideally wear off so that the organism is motivated to continue searching for more and better outcomes.²

Let us all take this lesson into our work in 2020 so that when it comes time to write this column next year in the chilling cold of late autumn there will be more stories of light than darkness from which to choose.

“For every complex problem there is a simple solution. And it’s wrong.”
—Anonymous as quoted in Barry Johnson’s Polarity Management¹

Hospital medicine leaders often face what seem like unsolvable problems involving two opposing sides or viewpoints. Examples include individual versus team, margin versus mission, learner autonomy versus supervision, and customization versus standardization. Dr. Barry Johnson describes these dyads as polarities, which are two different values or points of view that are interdependent.^1,2 Leaders who fail to realize this concept create a problem by artificially inserting the word ‘versus’ between the poles.

Polarities are not problems to be solved. How does one solve individual? Or team? How can a hospital have one without the other? When leaders treat polarities like problems to be solved, they typically crusade for one side over the other, until the losing side rises up for its own cause, causing a perpetual back and forth cycle described as an infinity loop where nobody is happy for long.¹

How then can leaders avoid getting caught in this fruitless cycle?

Instead of trying to solve the unsolvable, learn to manage polarities. Polarity management seeks to maximize the best of both poles while minimizing the worst. Both sides of a polarity carry upsides and downsides. When leaders want change, or want to resist change, it is the fear of being caught in the downsides of the opposite pole that motivates behavior, and dominates conversation. The first step to changing this conversation is to introduce the concept to your team so they recognize polarities when they arise and model approaching issues in this manner.

Some issues truly are problems to be solved (for example, the ultrasound machine is broken and needs to be repaired), but many conflicts are polarities masquerading as problems. To identify polarities, ask two questions. (1) Is the situation ongoing? (2) Are there two interdependent poles? If yes, then the issue is a polarity. Ideal polarity management involves maximizing the upside values of both poles before potential conflict even begins. People often force themselves into unnecessary “either/or” mindsets rather than striving for “both/and”.

Here is a classic example in Hospital Medicine: Pole 1: customization Pole 2: standardization -- The Chief Medical Information Officer (CMIO) wants everyone to use the same electronic health record (EHR) template, while the hospitalist group wants to innovate templates using rapid cycles of change. Typical patterns of conflict: the CMIO releases a template and the hospitalists resent it, or the hospitalists each create their own notes but the CMIO bemoans the variability.

Once polarities are recognized, teams can draw a ‘polarity map’ to see the whole picture, identifying the upside values and downside fears of each pole.^1,2 For example, standardization reduces unnecessary variation, but stifles innovation, while customization does the opposite. In fact, the upside values of one pole are usually the opposite of the downside fears of the other.

Leaders can actively engage people in both poles to make opposing views productive rather than destructive. The CMIO in our standardization/customization example could insist that everyone begins with the same template, but allow hospitalists to innovate to find a better way. Now the most resistant hospitalists become improvement agents. If a better way is found, then this becomes the new template that all hospitalists use, until the next better way is found. If an innovation is not an improvement, then hospitalists agree to return to the most recent successful template until a better way is found. This method of action and compromise produces both standardization and customization.

Using polarity management strategies does not guarantee success, but it can help engage all stakeholders, and break the frustrating cycle of repeatedly trying to solve the unsolvable.

Disclosures

The authors report no conflicts of interest or sources of funding.

“For every complex problem there is a simple solution. And it’s wrong.”
—Anonymous as quoted in Barry Johnson’s Polarity Management¹

Hospital medicine leaders often face what seem like unsolvable problems involving two opposing sides or viewpoints. Examples include individual versus team, margin versus mission, learner autonomy versus supervision, and customization versus standardization. Dr. Barry Johnson describes these dyads as polarities, which are two different values or points of view that are interdependent.^1,2 Leaders who fail to realize this concept create a problem by artificially inserting the word ‘versus’ between the poles.

Polarities are not problems to be solved. How does one solve individual? Or team? How can a hospital have one without the other? When leaders treat polarities like problems to be solved, they typically crusade for one side over the other, until the losing side rises up for its own cause, causing a perpetual back and forth cycle described as an infinity loop where nobody is happy for long.¹

How then can leaders avoid getting caught in this fruitless cycle?

Instead of trying to solve the unsolvable, learn to manage polarities. Polarity management seeks to maximize the best of both poles while minimizing the worst. Both sides of a polarity carry upsides and downsides. When leaders want change, or want to resist change, it is the fear of being caught in the downsides of the opposite pole that motivates behavior, and dominates conversation. The first step to changing this conversation is to introduce the concept to your team so they recognize polarities when they arise and model approaching issues in this manner.

Some issues truly are problems to be solved (for example, the ultrasound machine is broken and needs to be repaired), but many conflicts are polarities masquerading as problems. To identify polarities, ask two questions. (1) Is the situation ongoing? (2) Are there two interdependent poles? If yes, then the issue is a polarity. Ideal polarity management involves maximizing the upside values of both poles before potential conflict even begins. People often force themselves into unnecessary “either/or” mindsets rather than striving for “both/and”.

Here is a classic example in Hospital Medicine: Pole 1: customization Pole 2: standardization -- The Chief Medical Information Officer (CMIO) wants everyone to use the same electronic health record (EHR) template, while the hospitalist group wants to innovate templates using rapid cycles of change. Typical patterns of conflict: the CMIO releases a template and the hospitalists resent it, or the hospitalists each create their own notes but the CMIO bemoans the variability.

Once polarities are recognized, teams can draw a ‘polarity map’ to see the whole picture, identifying the upside values and downside fears of each pole.^1,2 For example, standardization reduces unnecessary variation, but stifles innovation, while customization does the opposite. In fact, the upside values of one pole are usually the opposite of the downside fears of the other.

Leaders can actively engage people in both poles to make opposing views productive rather than destructive. The CMIO in our standardization/customization example could insist that everyone begins with the same template, but allow hospitalists to innovate to find a better way. Now the most resistant hospitalists become improvement agents. If a better way is found, then this becomes the new template that all hospitalists use, until the next better way is found. If an innovation is not an improvement, then hospitalists agree to return to the most recent successful template until a better way is found. This method of action and compromise produces both standardization and customization.

Using polarity management strategies does not guarantee success, but it can help engage all stakeholders, and break the frustrating cycle of repeatedly trying to solve the unsolvable.

Disclosures

The authors report no conflicts of interest or sources of funding.

“For every complex problem there is a simple solution. And it’s wrong.”
—Anonymous as quoted in Barry Johnson’s Polarity Management¹

Hospital medicine leaders often face what seem like unsolvable problems involving two opposing sides or viewpoints. Examples include individual versus team, margin versus mission, learner autonomy versus supervision, and customization versus standardization. Dr. Barry Johnson describes these dyads as polarities, which are two different values or points of view that are interdependent.^1,2 Leaders who fail to realize this concept create a problem by artificially inserting the word ‘versus’ between the poles.

Polarities are not problems to be solved. How does one solve individual? Or team? How can a hospital have one without the other? When leaders treat polarities like problems to be solved, they typically crusade for one side over the other, until the losing side rises up for its own cause, causing a perpetual back and forth cycle described as an infinity loop where nobody is happy for long.¹

How then can leaders avoid getting caught in this fruitless cycle?

Instead of trying to solve the unsolvable, learn to manage polarities. Polarity management seeks to maximize the best of both poles while minimizing the worst. Both sides of a polarity carry upsides and downsides. When leaders want change, or want to resist change, it is the fear of being caught in the downsides of the opposite pole that motivates behavior, and dominates conversation. The first step to changing this conversation is to introduce the concept to your team so they recognize polarities when they arise and model approaching issues in this manner.

Some issues truly are problems to be solved (for example, the ultrasound machine is broken and needs to be repaired), but many conflicts are polarities masquerading as problems. To identify polarities, ask two questions. (1) Is the situation ongoing? (2) Are there two interdependent poles? If yes, then the issue is a polarity. Ideal polarity management involves maximizing the upside values of both poles before potential conflict even begins. People often force themselves into unnecessary “either/or” mindsets rather than striving for “both/and”.

Here is a classic example in Hospital Medicine: Pole 1: customization Pole 2: standardization -- The Chief Medical Information Officer (CMIO) wants everyone to use the same electronic health record (EHR) template, while the hospitalist group wants to innovate templates using rapid cycles of change. Typical patterns of conflict: the CMIO releases a template and the hospitalists resent it, or the hospitalists each create their own notes but the CMIO bemoans the variability.

Once polarities are recognized, teams can draw a ‘polarity map’ to see the whole picture, identifying the upside values and downside fears of each pole.^1,2 For example, standardization reduces unnecessary variation, but stifles innovation, while customization does the opposite. In fact, the upside values of one pole are usually the opposite of the downside fears of the other.

Leaders can actively engage people in both poles to make opposing views productive rather than destructive. The CMIO in our standardization/customization example could insist that everyone begins with the same template, but allow hospitalists to innovate to find a better way. Now the most resistant hospitalists become improvement agents. If a better way is found, then this becomes the new template that all hospitalists use, until the next better way is found. If an innovation is not an improvement, then hospitalists agree to return to the most recent successful template until a better way is found. This method of action and compromise produces both standardization and customization.

Using polarity management strategies does not guarantee success, but it can help engage all stakeholders, and break the frustrating cycle of repeatedly trying to solve the unsolvable.

Disclosures

The authors report no conflicts of interest or sources of funding.

Viral bronchiolitis is the most common indication for infant hospitalization in the United States.¹ The treatment mainstay remains supportive care, including supplemental oxygen when indicated.¹ High flow nasal cannula (HFNC) therapy delivers humidified, heated air blended with oxygen, allowing much higher flow rates than standard nasal cannula therapy and is being used more frequently in inpatient settings.

Infants and toddlers with bronchiolitis develop increased work of breathing to preserve oxygenation and ventilation in the setting of altered airway resistance and lung compliance.^2,3 In addition to oxygen supplementation, HFNC is used to reduce work of breathing through several mechanisms:^2-6 (1) Nasopharyngeal dead space washout clears oxygen-depleted gas at the end of expiration, facilitating alveolar ventilation (ie, carbon dioxide retention improves); (2) High flow rates match increased inspiratory flow demands of acutely ill patients, reducing nasopharyngeal inspiratory resistance and optimizing dead space washout, thus decreasing work of breathing; (3) Adequate flow rates generate distending pressure, which prevents pharyngeal collapse, supports lung recruitment, and reduces respiratory effort (demonstrated in younger infants); and (4) HFNC systems heat and humidify the breathing gas, reducing the metabolic work required to condition cool, dry gas and improving conductance and pulmonary compliance.^2-5

HFNC therapy is used more commonly in acute care units despite limited literature on its effectiveness outside the intensive care unit (ICU).^7,8 We asked the question, “Does use of HFNC therapy for infants with bronchiolitis hospitalized in acute care units result in improved outcomes when compared with standard nasal cannula oxygen therapy, including length of stay (LOS), oxygen therapy duration, and preventing escalations of care such as ICU transfer, positive pressure ventilation, and intubation?” Also, do published studies provide guidance for the initiation and management of HFNC? We focused our search on studies published in the last five years that included patients with bronchiolitis treated with HFNC outside the ICU; here, we review those studies most relevant to pediatric hospitalists.

No guideline exists for initiating flow or fraction of inspired oxygen (FiO₂). HFNC may be initiated for hypoxia, increased work of breathing, or both in patients with bronchiolitis. To achieve optimal dead space washout, inspiratory flow, and distending pressure, initial flow rates should be 1.5 to 2 L/kg/min, particularly for infants and young children.^2-5 Weiler et al.³ evaluated the breathing effort of ICU patients at 0.5, 1, 1.5, and 2 L/kg/min and found optimal flow rates for improved work of breathing were 1.5-2 L/kg/min. The smallest patients, ≤8 kg, saw the greatest benefit, a finding likely explained by larger anatomic dead space in infants/small children compared with older children.³ For older/larger children (>20 kg), an initial flow closer to 1 L/kg/min is often appropriate.⁵ When used for hypoxia, initiating flow without supplemental FiO₂ may improve oxygenation by flushing nasopharyngeal dead space. FiO₂ should be titrated to achieve the goal set by the treatment team, often ≥90%. Improvement in heart rate and peripheral oxygen saturation (SpO₂) can be observed within 60 minutes of initiating HFNC in patients responsive to therapy.⁶

HFNC therapy is safe when used correctly.^6,9,10Potential adverse effects include pneumothorax, pressure injury, mucosal injury/bleeding, and delayed escalation to invasive ventilation. While difficult to quantify, recent studies report low rates or no serious HFNC complications. For example, only 2 of 1,127 patients supported with HFNC developed a pneumothorax and neither required evacuation.^2,9-12

Inclusion criteria and HFNC protocols vary among published studies. Most HFNC protocols reviewed may not have optimally supported all of the patients in their HFNC groups, often by limiting flow to <2 L/kg/min.^6-9,11,12 These variables may explain the disparate results, with some studies demonstrating apparent benefits and others no difference.^7,9,10,12

Two studies of infants with bronchiolitis showed HFNC therapy may prevent ICU transfer, but this benefit may be limited to rescue when standard oxygen therapy fails, rather than as a superior initial support modality.^7,9 Kepreotes et al.⁹ reported a single-center, randomized controlled trial comparing HFNC with standard oxygen therapy with 101 patients in each treatment arm. The primary outcome, median time to wean off oxygen, was not significantly different between the two groups: 24 hours (95% CI: 18-28) in the HFNC group versus 20 hours in the standard therapy group (95% CI: 17-34). The HFNC group had fewer treatment failures (abnormal heart rate, respiratory rate, SpO₂ <90%, or severe respiratory distress score while on maximum therapy) than the standard therapy group, and 20 (63%) of the 33 patients who failed standard therapy were rescued with HFNC, avoiding transfer to the ICU. Fourteen patients from the HFNC group and 12 from the standard oxygen group required transfer to the ICU for support escalation. Although this study did not show a significant difference in oxygen weaning time between groups, it appears to support HFNC use as a rescue modality to reduce or prevent ICU transfer.⁹ Franklin et al.¹⁰ conducted a multicenter, randomized, controlled trial to compare standard nasal cannula oxygen therapy with HFNC (2 L/kg/min) in 1,472 patients. Patients receiving HFNC had lower care escalation rates due to treatment failure, defined as the presence of at least three of four clinical criteria and the clinician determining escalation was indicated. Oxygen therapy duration, ICU admission rates, and LOS were not significantly different between groups. Similar to the previous study, a large portion of the standard therapy patients who failed treatment (102 of 167) crossed over to the HFNC arm in an attempt to avoid ICU transfer. Twelve patients required intubation: 8 (1%) receiving HFNC and 4 (0.5%) receiving the standard therapy.¹⁰

Two additional studies, both with study design limitations, did not demonstrate differences in ICU transfer rates and had variable differences in outcomes. Riese et al.⁷ retrospectively assessed HFNC use outside the ICU at one institution and included 936 patients admitted before and 1,001 patients admitted after HFNC guideline implementation on the wards. Flow rates were based on age and not weight. They found no difference in LOS, ICU transfer rate, ICU LOS, intubation rates, or 30-day readmission rates, though HFNC use increased over time. The HFNC guideline is a potentially significant limitation as it may not have provided optimal flow rates to all subjects given it was based on age rather than weight. Milani et al.¹² performed a single-center observational study of 36 infants aged <12 months, treated for bronchiolitis on the ward, who were informally assigned to HFNC or standard therapy based upon HFNC device availability. HFNC flow rate was determined by the equation: L/min = 8 mL/kg × respiratory rate × 0.3. Using mean weight and respiratory rate for patients in this group, it appears patients in the HFNC group were treated with flow rates less than the 1.5-2 L/kg/min recommended to be effective.^2,3,12 Despite this, clinical improvement was faster in the HFNC group, including respiratory rate and effort, ability to feed, days on oxygen supplementation, and hospital LOS. ICU admission was not different between the two groups.¹² The Table compares the four studies discussed above.

Many children’s hospitals have extended the use of HFNC outside the ICU for children with bronchiolitis despite the paucity of evidence demonstrating its benefit over standard flow oxygen. Given variation in protocols, study designs, outcomes, and number of patients studied, it is difficult to assess its efficacy outside the ICU. However, based on the studies reviewed herein, HFNC therapy does not appear to decrease LOS, time on oxygen, or escalations of care, such as ICU transfers, positive pressure ventilation, or intubation, when used as a primary therapy.^7,9,11,12 Future research will ideally use optimal flow rates to determine the effectiveness of HFNC on acute care units. Although not addressed in the above studies, additional benefits to be considered in future studies include: (1) increased critical care capacity by allowing patients to be supported on the floor and (2) the ability for patients to remain closer to home when HFNC is used in the community hospital setting.

In each of the large, randomized studies reviewed, most (66%-75%) patients treated with standard low flow oxygen were supported successfully and did not require escalation to HFNC.^9,10 Hospitalists should continue to use standard low flow oxygen as first-line respiratory support for patients with bronchiolitis.¹ No evidence supports the use of HFNC therapy early in a child’s inpatient course; rather, it should be used when standard oxygen therapy fails. Future research should focus on better elucidating which patients will benefit most from HFNC to prevent overuse.

Viral bronchiolitis is the most common indication for infant hospitalization in the United States.¹ The treatment mainstay remains supportive care, including supplemental oxygen when indicated.¹ High flow nasal cannula (HFNC) therapy delivers humidified, heated air blended with oxygen, allowing much higher flow rates than standard nasal cannula therapy and is being used more frequently in inpatient settings.

Infants and toddlers with bronchiolitis develop increased work of breathing to preserve oxygenation and ventilation in the setting of altered airway resistance and lung compliance.^2,3 In addition to oxygen supplementation, HFNC is used to reduce work of breathing through several mechanisms:^2-6 (1) Nasopharyngeal dead space washout clears oxygen-depleted gas at the end of expiration, facilitating alveolar ventilation (ie, carbon dioxide retention improves); (2) High flow rates match increased inspiratory flow demands of acutely ill patients, reducing nasopharyngeal inspiratory resistance and optimizing dead space washout, thus decreasing work of breathing; (3) Adequate flow rates generate distending pressure, which prevents pharyngeal collapse, supports lung recruitment, and reduces respiratory effort (demonstrated in younger infants); and (4) HFNC systems heat and humidify the breathing gas, reducing the metabolic work required to condition cool, dry gas and improving conductance and pulmonary compliance.^2-5

HFNC therapy is used more commonly in acute care units despite limited literature on its effectiveness outside the intensive care unit (ICU).^7,8 We asked the question, “Does use of HFNC therapy for infants with bronchiolitis hospitalized in acute care units result in improved outcomes when compared with standard nasal cannula oxygen therapy, including length of stay (LOS), oxygen therapy duration, and preventing escalations of care such as ICU transfer, positive pressure ventilation, and intubation?” Also, do published studies provide guidance for the initiation and management of HFNC? We focused our search on studies published in the last five years that included patients with bronchiolitis treated with HFNC outside the ICU; here, we review those studies most relevant to pediatric hospitalists.

No guideline exists for initiating flow or fraction of inspired oxygen (FiO₂). HFNC may be initiated for hypoxia, increased work of breathing, or both in patients with bronchiolitis. To achieve optimal dead space washout, inspiratory flow, and distending pressure, initial flow rates should be 1.5 to 2 L/kg/min, particularly for infants and young children.^2-5 Weiler et al.³ evaluated the breathing effort of ICU patients at 0.5, 1, 1.5, and 2 L/kg/min and found optimal flow rates for improved work of breathing were 1.5-2 L/kg/min. The smallest patients, ≤8 kg, saw the greatest benefit, a finding likely explained by larger anatomic dead space in infants/small children compared with older children.³ For older/larger children (>20 kg), an initial flow closer to 1 L/kg/min is often appropriate.⁵ When used for hypoxia, initiating flow without supplemental FiO₂ may improve oxygenation by flushing nasopharyngeal dead space. FiO₂ should be titrated to achieve the goal set by the treatment team, often ≥90%. Improvement in heart rate and peripheral oxygen saturation (SpO₂) can be observed within 60 minutes of initiating HFNC in patients responsive to therapy.⁶

HFNC therapy is safe when used correctly.^6,9,10Potential adverse effects include pneumothorax, pressure injury, mucosal injury/bleeding, and delayed escalation to invasive ventilation. While difficult to quantify, recent studies report low rates or no serious HFNC complications. For example, only 2 of 1,127 patients supported with HFNC developed a pneumothorax and neither required evacuation.^2,9-12

Inclusion criteria and HFNC protocols vary among published studies. Most HFNC protocols reviewed may not have optimally supported all of the patients in their HFNC groups, often by limiting flow to <2 L/kg/min.^6-9,11,12 These variables may explain the disparate results, with some studies demonstrating apparent benefits and others no difference.^7,9,10,12

Two studies of infants with bronchiolitis showed HFNC therapy may prevent ICU transfer, but this benefit may be limited to rescue when standard oxygen therapy fails, rather than as a superior initial support modality.^7,9 Kepreotes et al.⁹ reported a single-center, randomized controlled trial comparing HFNC with standard oxygen therapy with 101 patients in each treatment arm. The primary outcome, median time to wean off oxygen, was not significantly different between the two groups: 24 hours (95% CI: 18-28) in the HFNC group versus 20 hours in the standard therapy group (95% CI: 17-34). The HFNC group had fewer treatment failures (abnormal heart rate, respiratory rate, SpO₂ <90%, or severe respiratory distress score while on maximum therapy) than the standard therapy group, and 20 (63%) of the 33 patients who failed standard therapy were rescued with HFNC, avoiding transfer to the ICU. Fourteen patients from the HFNC group and 12 from the standard oxygen group required transfer to the ICU for support escalation. Although this study did not show a significant difference in oxygen weaning time between groups, it appears to support HFNC use as a rescue modality to reduce or prevent ICU transfer.⁹ Franklin et al.¹⁰ conducted a multicenter, randomized, controlled trial to compare standard nasal cannula oxygen therapy with HFNC (2 L/kg/min) in 1,472 patients. Patients receiving HFNC had lower care escalation rates due to treatment failure, defined as the presence of at least three of four clinical criteria and the clinician determining escalation was indicated. Oxygen therapy duration, ICU admission rates, and LOS were not significantly different between groups. Similar to the previous study, a large portion of the standard therapy patients who failed treatment (102 of 167) crossed over to the HFNC arm in an attempt to avoid ICU transfer. Twelve patients required intubation: 8 (1%) receiving HFNC and 4 (0.5%) receiving the standard therapy.¹⁰

Two additional studies, both with study design limitations, did not demonstrate differences in ICU transfer rates and had variable differences in outcomes. Riese et al.⁷ retrospectively assessed HFNC use outside the ICU at one institution and included 936 patients admitted before and 1,001 patients admitted after HFNC guideline implementation on the wards. Flow rates were based on age and not weight. They found no difference in LOS, ICU transfer rate, ICU LOS, intubation rates, or 30-day readmission rates, though HFNC use increased over time. The HFNC guideline is a potentially significant limitation as it may not have provided optimal flow rates to all subjects given it was based on age rather than weight. Milani et al.¹² performed a single-center observational study of 36 infants aged <12 months, treated for bronchiolitis on the ward, who were informally assigned to HFNC or standard therapy based upon HFNC device availability. HFNC flow rate was determined by the equation: L/min = 8 mL/kg × respiratory rate × 0.3. Using mean weight and respiratory rate for patients in this group, it appears patients in the HFNC group were treated with flow rates less than the 1.5-2 L/kg/min recommended to be effective.^2,3,12 Despite this, clinical improvement was faster in the HFNC group, including respiratory rate and effort, ability to feed, days on oxygen supplementation, and hospital LOS. ICU admission was not different between the two groups.¹² The Table compares the four studies discussed above.

Many children’s hospitals have extended the use of HFNC outside the ICU for children with bronchiolitis despite the paucity of evidence demonstrating its benefit over standard flow oxygen. Given variation in protocols, study designs, outcomes, and number of patients studied, it is difficult to assess its efficacy outside the ICU. However, based on the studies reviewed herein, HFNC therapy does not appear to decrease LOS, time on oxygen, or escalations of care, such as ICU transfers, positive pressure ventilation, or intubation, when used as a primary therapy.^7,9,11,12 Future research will ideally use optimal flow rates to determine the effectiveness of HFNC on acute care units. Although not addressed in the above studies, additional benefits to be considered in future studies include: (1) increased critical care capacity by allowing patients to be supported on the floor and (2) the ability for patients to remain closer to home when HFNC is used in the community hospital setting.

In each of the large, randomized studies reviewed, most (66%-75%) patients treated with standard low flow oxygen were supported successfully and did not require escalation to HFNC.^9,10 Hospitalists should continue to use standard low flow oxygen as first-line respiratory support for patients with bronchiolitis.¹ No evidence supports the use of HFNC therapy early in a child’s inpatient course; rather, it should be used when standard oxygen therapy fails. Future research should focus on better elucidating which patients will benefit most from HFNC to prevent overuse.

Viral bronchiolitis is the most common indication for infant hospitalization in the United States.¹ The treatment mainstay remains supportive care, including supplemental oxygen when indicated.¹ High flow nasal cannula (HFNC) therapy delivers humidified, heated air blended with oxygen, allowing much higher flow rates than standard nasal cannula therapy and is being used more frequently in inpatient settings.

Infants and toddlers with bronchiolitis develop increased work of breathing to preserve oxygenation and ventilation in the setting of altered airway resistance and lung compliance.^2,3 In addition to oxygen supplementation, HFNC is used to reduce work of breathing through several mechanisms:^2-6 (1) Nasopharyngeal dead space washout clears oxygen-depleted gas at the end of expiration, facilitating alveolar ventilation (ie, carbon dioxide retention improves); (2) High flow rates match increased inspiratory flow demands of acutely ill patients, reducing nasopharyngeal inspiratory resistance and optimizing dead space washout, thus decreasing work of breathing; (3) Adequate flow rates generate distending pressure, which prevents pharyngeal collapse, supports lung recruitment, and reduces respiratory effort (demonstrated in younger infants); and (4) HFNC systems heat and humidify the breathing gas, reducing the metabolic work required to condition cool, dry gas and improving conductance and pulmonary compliance.^2-5

HFNC therapy is used more commonly in acute care units despite limited literature on its effectiveness outside the intensive care unit (ICU).^7,8 We asked the question, “Does use of HFNC therapy for infants with bronchiolitis hospitalized in acute care units result in improved outcomes when compared with standard nasal cannula oxygen therapy, including length of stay (LOS), oxygen therapy duration, and preventing escalations of care such as ICU transfer, positive pressure ventilation, and intubation?” Also, do published studies provide guidance for the initiation and management of HFNC? We focused our search on studies published in the last five years that included patients with bronchiolitis treated with HFNC outside the ICU; here, we review those studies most relevant to pediatric hospitalists.

No guideline exists for initiating flow or fraction of inspired oxygen (FiO₂). HFNC may be initiated for hypoxia, increased work of breathing, or both in patients with bronchiolitis. To achieve optimal dead space washout, inspiratory flow, and distending pressure, initial flow rates should be 1.5 to 2 L/kg/min, particularly for infants and young children.^2-5 Weiler et al.³ evaluated the breathing effort of ICU patients at 0.5, 1, 1.5, and 2 L/kg/min and found optimal flow rates for improved work of breathing were 1.5-2 L/kg/min. The smallest patients, ≤8 kg, saw the greatest benefit, a finding likely explained by larger anatomic dead space in infants/small children compared with older children.³ For older/larger children (>20 kg), an initial flow closer to 1 L/kg/min is often appropriate.⁵ When used for hypoxia, initiating flow without supplemental FiO₂ may improve oxygenation by flushing nasopharyngeal dead space. FiO₂ should be titrated to achieve the goal set by the treatment team, often ≥90%. Improvement in heart rate and peripheral oxygen saturation (SpO₂) can be observed within 60 minutes of initiating HFNC in patients responsive to therapy.⁶

HFNC therapy is safe when used correctly.^6,9,10Potential adverse effects include pneumothorax, pressure injury, mucosal injury/bleeding, and delayed escalation to invasive ventilation. While difficult to quantify, recent studies report low rates or no serious HFNC complications. For example, only 2 of 1,127 patients supported with HFNC developed a pneumothorax and neither required evacuation.^2,9-12

Inclusion criteria and HFNC protocols vary among published studies. Most HFNC protocols reviewed may not have optimally supported all of the patients in their HFNC groups, often by limiting flow to <2 L/kg/min.^6-9,11,12 These variables may explain the disparate results, with some studies demonstrating apparent benefits and others no difference.^7,9,10,12

Two studies of infants with bronchiolitis showed HFNC therapy may prevent ICU transfer, but this benefit may be limited to rescue when standard oxygen therapy fails, rather than as a superior initial support modality.^7,9 Kepreotes et al.⁹ reported a single-center, randomized controlled trial comparing HFNC with standard oxygen therapy with 101 patients in each treatment arm. The primary outcome, median time to wean off oxygen, was not significantly different between the two groups: 24 hours (95% CI: 18-28) in the HFNC group versus 20 hours in the standard therapy group (95% CI: 17-34). The HFNC group had fewer treatment failures (abnormal heart rate, respiratory rate, SpO₂ <90%, or severe respiratory distress score while on maximum therapy) than the standard therapy group, and 20 (63%) of the 33 patients who failed standard therapy were rescued with HFNC, avoiding transfer to the ICU. Fourteen patients from the HFNC group and 12 from the standard oxygen group required transfer to the ICU for support escalation. Although this study did not show a significant difference in oxygen weaning time between groups, it appears to support HFNC use as a rescue modality to reduce or prevent ICU transfer.⁹ Franklin et al.¹⁰ conducted a multicenter, randomized, controlled trial to compare standard nasal cannula oxygen therapy with HFNC (2 L/kg/min) in 1,472 patients. Patients receiving HFNC had lower care escalation rates due to treatment failure, defined as the presence of at least three of four clinical criteria and the clinician determining escalation was indicated. Oxygen therapy duration, ICU admission rates, and LOS were not significantly different between groups. Similar to the previous study, a large portion of the standard therapy patients who failed treatment (102 of 167) crossed over to the HFNC arm in an attempt to avoid ICU transfer. Twelve patients required intubation: 8 (1%) receiving HFNC and 4 (0.5%) receiving the standard therapy.¹⁰

Two additional studies, both with study design limitations, did not demonstrate differences in ICU transfer rates and had variable differences in outcomes. Riese et al.⁷ retrospectively assessed HFNC use outside the ICU at one institution and included 936 patients admitted before and 1,001 patients admitted after HFNC guideline implementation on the wards. Flow rates were based on age and not weight. They found no difference in LOS, ICU transfer rate, ICU LOS, intubation rates, or 30-day readmission rates, though HFNC use increased over time. The HFNC guideline is a potentially significant limitation as it may not have provided optimal flow rates to all subjects given it was based on age rather than weight. Milani et al.¹² performed a single-center observational study of 36 infants aged <12 months, treated for bronchiolitis on the ward, who were informally assigned to HFNC or standard therapy based upon HFNC device availability. HFNC flow rate was determined by the equation: L/min = 8 mL/kg × respiratory rate × 0.3. Using mean weight and respiratory rate for patients in this group, it appears patients in the HFNC group were treated with flow rates less than the 1.5-2 L/kg/min recommended to be effective.^2,3,12 Despite this, clinical improvement was faster in the HFNC group, including respiratory rate and effort, ability to feed, days on oxygen supplementation, and hospital LOS. ICU admission was not different between the two groups.¹² The Table compares the four studies discussed above.

Many children’s hospitals have extended the use of HFNC outside the ICU for children with bronchiolitis despite the paucity of evidence demonstrating its benefit over standard flow oxygen. Given variation in protocols, study designs, outcomes, and number of patients studied, it is difficult to assess its efficacy outside the ICU. However, based on the studies reviewed herein, HFNC therapy does not appear to decrease LOS, time on oxygen, or escalations of care, such as ICU transfers, positive pressure ventilation, or intubation, when used as a primary therapy.^7,9,11,12 Future research will ideally use optimal flow rates to determine the effectiveness of HFNC on acute care units. Although not addressed in the above studies, additional benefits to be considered in future studies include: (1) increased critical care capacity by allowing patients to be supported on the floor and (2) the ability for patients to remain closer to home when HFNC is used in the community hospital setting.

In each of the large, randomized studies reviewed, most (66%-75%) patients treated with standard low flow oxygen were supported successfully and did not require escalation to HFNC.^9,10 Hospitalists should continue to use standard low flow oxygen as first-line respiratory support for patients with bronchiolitis.¹ No evidence supports the use of HFNC therapy early in a child’s inpatient course; rather, it should be used when standard oxygen therapy fails. Future research should focus on better elucidating which patients will benefit most from HFNC to prevent overuse.

High-flow nasal cannula (HFNC) oxygen therapy is effective in treating adults with acute hypoxemic respiratory failure, and to a lesser extent acute hypercapnic respiratory failure.^1-3 HFNC oxygen is capable of delivering oxygen with flows of 30-60 liters/minute, and can provide a high fraction of inspired oxygen, flush anatomic dead space, augment respiratory efforts, and provide mild continuous positive airway pressure effects. Several systematic reviews and meta-analyses have evaluated the effectiveness of HFNC oxygen and have shown modestly lower rates of intubation compared with conventional oxygen^4,5 and similar intubation rates compared with noninvasive positive pressure ventilation.^4-9 Although one randomized trial showed a lower risk of 90-day mortality for HFNC oxygen compared with either conventional oxygen or noninvasive positive pressure ventilation, several meta-analyses have shown no difference in intensive care unit (ICU) mortality.^4,6,8,10 The majority of studies have shown improvements in oxygenation, comfort, dyspnea scores, and breathing pattern with the initiation of HFNC oxygen.⁶

While the evidence to support the use of HFNC oxygen in patients with nonhypercapnic acute hypoxemic respiratory failure is growing, this evidence is based on patients enrolled in clinical trials who have no treatment limitations and consent to intubation if necessary. Indeed, several, if not all, randomized trials evaluating HFNC oxygen excluded patients who had do-not-intubate (DNI) or do-not-resuscitate (DNR) orders.^1,2,11 For patients with acute respiratory failure whose primary goal is not to extend life or utilize life support interventions such as invasive mechanical ventilation, HFNC oxygen may offer several benefits compared with other treatment options such as noninvasive positive pressure ventilation, conventional oxygen therapy, or palliative opioid therapy (Appendix Table 1). Determining which treatment options to use depends on the goals of care of the individual patient and the reasonable ability of a particular treatment to help the patient achieve those goals.

While a recent systematic review evaluated the existing evidence regarding the utility and outcomes of noninvasive positive pressure ventilation in adult patients with DNI orders,¹² a systematic review evaluating the evidence and rationale for HFNC oxygen in patients with DNI and/or DNR orders is lacking. Assessing such evidence is necessary to help clinicians and patients determine appropriate treatment choices and establish research priorities. Therefore, our primary objective was to determine what were the following outcomes: mortality, dyspnea, work of breathing, opioid doses, and quality of life in patients who received HFNC oxygen for acute respiratory failure and had a DNI and/or DNR order.

We conducted a systematic review of studies that evaluated patients who used HFNC oxygen for acute respiratory failure and had a DNI and/or DNR order. We reported the results using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statements.¹³ This review was registered with the PROSPERO registry, CRD42017059914.

We included studies that enrolled patients who were (1) hospitalized, (2) >18 years old, (3) had an acute respiratory failure of any cause, (4) received HFNC oxygen, and (5) had a DNI or DNR or comfort measures only order. We included publications of all study designs (interventional, observational, and posthoc analyses) and all languages. We excluded studies that enrolled <5 patients. If necessary, we contacted the authors of the included studies for additional information.

Our search strategy included the following databases from inception to October 14, 2018: PubMed, MEDLINE, CINAHL, MICROMEDEX, EMBASE, Web of Science, and Scopus. The database-specific search strategy was developed using an experienced librarian (Appendix Table 2). In addition, we screened the reference lists of systematic reviews as well as the included studies to find additional relevant articles. Two authors (AM, MEW) independently assessed the inclusion criteria of the titles and abstracts that were identified in the search. In addition, these two authors abstracted relevant data of the included studies.

Using the search strategy, we identified 2,757 citations and included 301 of these in the full-text review (Figure). We included six studies, which enrolled 293 patients in the final systematic review. Table 1 summarizes the characteristics of the included investigations, all of which were observational studies.^15-20 The studies were conducted in the United States of America (n = 3), Europe (n = 2), and Asia (n = 1). Two studies were conducted in the general ICU populations and included patients with hypoxemic respiratory failure only. Four studies were conducted in cancer populations in the hospital wards or ICU and did not specify the type of respiratory failure (hypoxemic versus hypercapnic). Two studies included patients with DNI orders only.^15,20 One study included patients with DNR orders only (DNI orders were excluded).¹⁷ Three studies included patients with both DNR and DNI orders.^16,18,19 The numbers of enrolled patients with treatment limitations were generally low, with the two largest studies including 101 patients each on HFNC oxygen.^18,19

Risk of Bias

All included studies had a high risk of bias (Table 2). A high risk of bias was suggested because the investigations were single-center studies with unclear patient selection methods, did not explicitly report how decisions to limit treatments were made, and did not explicitly differentiate and separately analyze patients with “comfort measures only” goals of care.

Mortality

The hospital mortality rates of patients with DNI and/or DNR orders receiving HFNC were variable and ranged from 40% to 87%. In the two studies enrolling general ICU patient populations, the hospital mortality rates ranged from 40% to 60%. In the four studies enrolling patients with active malignancy, the hospital mortality rates ranged from 75% to 87%. No studies compared mortality rates with and without DNI and/or DNR orders.

Dyspnea, Work of Breathing, and Reduction in Opioid Doses

The impact of HFNC oxygen on symptom relief was reported in one retrospective observational study (published as a conference abstract only to date), which compared the effect of HFNC oxygen (n = 101) with conventional oxygen (n = 110).¹⁸ At first evaluation after hospital admission to a palliative care unit (after the patients had previously been started on either conventional oxygen or high-flow oxygen), patients in the HFNC oxygen group had worse (higher) dyspnea scores compared with patients who used conventional oxygen (Edmonton Symptom Assessment Scale score of 7.5 versus 5, P < .001). At follow-up, approximately 24 hours after admission to the hospital palliative care unit, there was no difference in the change of dyspnea between the HFNC oxygen group (dyspnea score change of 0) and the conventional oxygen group (dyspnea score change of −1, P = .18. In the same study, there was also no significant difference in the morphine dose requirement in each group, and exact doses were not reported.

Two studies reported improvement in oxygen saturation and respiratory rate after HFNC oxygen initiation (compared with before HFNC initiation).^16,20 Oxygen saturation increased from 89% to 95%, P < .01, in one study and 92% to 97%, P < .01, in a second study. The respiratory rate decreased from 31 to 25 breaths/minute in one study, and from 28 to 25 breaths/minute in a second study (both P < .01).

Quality of Life

No studies evaluated the quality of life of survivors.

Secondary Outcomes

Transition to Noninvasive Positive Pressure Ventilation

The proportion of patients who transitioned from HFNC oxygen to NPPV was relatively low in the two studies that reported this outcome, ranging from 0%²⁰ to 18%.¹⁶ In one observational study of a general ICU population, 9/50 (18%) of patients transitioned from HFNC oxygen to NPPV. There was no statistically significant difference in hospital mortality rates among those who progressed to NPPV (67%) versus those who did not progress to NPPV (58%), P = .72.

Tolerance of HFNC and Adverse Events

HFNC oxygen was generally well tolerated based on the assessment of three studies (Table 1). One study reported no adverse events,¹⁶ one study reported that HFNC oxygen had to be discontinued because of nasal discomfort in 1% of patients,¹⁹ and a second study reported that HFNC oxygen had to be discontinued because of agitation in 4% of patients.²⁰

Quality of Death in Nonsurvivors

No studies evaluated the quality of death in those patients who died.

In this systematic review of six studies, all with a high risk of bias, a significant proportion of patients with a DNI and/or DNR order who used HFNC oxygen survived to hospital discharge. Oxygen saturation and respiratory rate consistently improved in the three studies that reported these outcomes. Only one study (published as a conference abstract only to date),¹⁸ however, measured patient-important outcomes related to symptom management and found no significant difference in dyspnea or morphine dose requirements in patients on HFNC oxygen compared with patients on conventional oxygen. HFNC oxygen was generally well tolerated and only had to be stopped in <5% of patients due to intolerance. We found no studies that assessed the quality of life in survivors or the quality of death in nonsurvivors.

Based on the limited evidence in the included studies, HFNC may be a viable treatment option for patients with preset treatment limitations who have acute respiratory failure—with potential benefits of improved oxygenation, decreased respiratory rates, and hospital survival in a proportion of patients. Nevertheless, this systematic review highlights the vast paucity of data available to guide the use of HFNC oxygen in patients with treatment limitations and acute respiratory failure. Only a few studies, which were at high risk of bias, have been conducted on this topic to date. There is an inadequate evidence base to evaluate the comparative effectiveness of HFNC oxygen (versus NPPV versus conventional oxygen versus palliative opioids) in patients with DNI orders or comfort measures only orders.

Our review included two studies that evaluated the comparative effectiveness of HFNC oxygen in patients with DNI and/or DNR orders. The first retrospective observational study compared HFNC oxygen with conventional oxygen in patients with DNR and DNI orders and malignancy—and found no change in dyspnea—but did note an increase in mortality with HFNC oxygen (76% versus 51%).¹⁸ The second observational study compared HFNC oxygen with NPPV in patients with DNR orders with malignancy noted no difference in mortality.¹⁷ In patients with full-code orders, systematic reviews have shown that HFNC oxygen (compared with conventional oxygen) was associated with possible reductions in intubation rates, respiratory rates, and improvements in oxygenation—with no difference in mortality, dyspnea, patient comfort, or ICU/hospital length of stay. Compared with NPPV, HFNC oxygen was associated with similar rates of intubation and mortality.^4-6,21

Future studies in patients with acute respiratory failure and DNI and/or DNR orders should identify which treatment modality (HFNC oxygen compared with other modalities, such as NPPV, conventional oxygen, with or without palliative opioids) impacts outcomes, such as dyspnea reduction while maintaining an alert mental status, short- and long-term quality of life in survivors, and quality of death in nonsurvivors. Future studies should also identify the optimal treatment pathway to utilize when patients using HFNC oxygen fail this therapy (eg, transition to NPPV versus intensifying palliative opioids) as well as the optimal process to withdraw palliative HFNC oxygen.²² Identifying which patient populations may benefit from different treatment pathways should also be considered as different treatment strategies may be more beneficial in different patient populations (eg, based on cause and severity of acute respiratory failure). In addition, it should be noted that the primary goal of care might affect which outcomes are the most important to measure. While patients with comfort measures only, orders usually have a primary goal to prepare for a high-quality death, patients with DNI and/or DNR orders (but without comfort measures only orders) may have a primary goal to survive—but with the desire not to endure the high burden of intubation and mechanical ventilation if it became necessary. Finally, future studies should utilize high-quality study designs (eg, randomized controlled trials) that enable robust evaluation of comparative effectiveness of clinically relevant treatment strategies.

While several previous systematic reviews have evaluated the efficacy of HFNC in patients with acute respiratory failure without preset limitations on life support; to our knowledge, this is the first systematic review to assess outcomes in patients rigorously with preset treatment limitations. Our review is, however, limited by the high risk of bias of the studies that were included (single-center nature, retrospective observational study designs, small sample sizes, and lack of a description of how DNI and/or DNR statuses were determined) as well as the small number of studies available to be included.

This systematic review points to a significant evidence gap in our understanding of the role for HFNC oxygen (compared with other acceptable alternative treatment strategies) in adult patients with acute respiratory failure who have DNI and/or DNR orders. Further high-quality research is needed to explore these unanswered questions in an effort to best treat, guide, and engage in optimal end-of-life decision making among patients with acute respiratory failure.

High-flow nasal cannula (HFNC) oxygen therapy is effective in treating adults with acute hypoxemic respiratory failure, and to a lesser extent acute hypercapnic respiratory failure.^1-3 HFNC oxygen is capable of delivering oxygen with flows of 30-60 liters/minute, and can provide a high fraction of inspired oxygen, flush anatomic dead space, augment respiratory efforts, and provide mild continuous positive airway pressure effects. Several systematic reviews and meta-analyses have evaluated the effectiveness of HFNC oxygen and have shown modestly lower rates of intubation compared with conventional oxygen^4,5 and similar intubation rates compared with noninvasive positive pressure ventilation.^4-9 Although one randomized trial showed a lower risk of 90-day mortality for HFNC oxygen compared with either conventional oxygen or noninvasive positive pressure ventilation, several meta-analyses have shown no difference in intensive care unit (ICU) mortality.^4,6,8,10 The majority of studies have shown improvements in oxygenation, comfort, dyspnea scores, and breathing pattern with the initiation of HFNC oxygen.⁶

While the evidence to support the use of HFNC oxygen in patients with nonhypercapnic acute hypoxemic respiratory failure is growing, this evidence is based on patients enrolled in clinical trials who have no treatment limitations and consent to intubation if necessary. Indeed, several, if not all, randomized trials evaluating HFNC oxygen excluded patients who had do-not-intubate (DNI) or do-not-resuscitate (DNR) orders.^1,2,11 For patients with acute respiratory failure whose primary goal is not to extend life or utilize life support interventions such as invasive mechanical ventilation, HFNC oxygen may offer several benefits compared with other treatment options such as noninvasive positive pressure ventilation, conventional oxygen therapy, or palliative opioid therapy (Appendix Table 1). Determining which treatment options to use depends on the goals of care of the individual patient and the reasonable ability of a particular treatment to help the patient achieve those goals.

While a recent systematic review evaluated the existing evidence regarding the utility and outcomes of noninvasive positive pressure ventilation in adult patients with DNI orders,¹² a systematic review evaluating the evidence and rationale for HFNC oxygen in patients with DNI and/or DNR orders is lacking. Assessing such evidence is necessary to help clinicians and patients determine appropriate treatment choices and establish research priorities. Therefore, our primary objective was to determine what were the following outcomes: mortality, dyspnea, work of breathing, opioid doses, and quality of life in patients who received HFNC oxygen for acute respiratory failure and had a DNI and/or DNR order.

We conducted a systematic review of studies that evaluated patients who used HFNC oxygen for acute respiratory failure and had a DNI and/or DNR order. We reported the results using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statements.¹³ This review was registered with the PROSPERO registry, CRD42017059914.

We included studies that enrolled patients who were (1) hospitalized, (2) >18 years old, (3) had an acute respiratory failure of any cause, (4) received HFNC oxygen, and (5) had a DNI or DNR or comfort measures only order. We included publications of all study designs (interventional, observational, and posthoc analyses) and all languages. We excluded studies that enrolled <5 patients. If necessary, we contacted the authors of the included studies for additional information.

Our search strategy included the following databases from inception to October 14, 2018: PubMed, MEDLINE, CINAHL, MICROMEDEX, EMBASE, Web of Science, and Scopus. The database-specific search strategy was developed using an experienced librarian (Appendix Table 2). In addition, we screened the reference lists of systematic reviews as well as the included studies to find additional relevant articles. Two authors (AM, MEW) independently assessed the inclusion criteria of the titles and abstracts that were identified in the search. In addition, these two authors abstracted relevant data of the included studies.

Using the search strategy, we identified 2,757 citations and included 301 of these in the full-text review (Figure). We included six studies, which enrolled 293 patients in the final systematic review. Table 1 summarizes the characteristics of the included investigations, all of which were observational studies.^15-20 The studies were conducted in the United States of America (n = 3), Europe (n = 2), and Asia (n = 1). Two studies were conducted in the general ICU populations and included patients with hypoxemic respiratory failure only. Four studies were conducted in cancer populations in the hospital wards or ICU and did not specify the type of respiratory failure (hypoxemic versus hypercapnic). Two studies included patients with DNI orders only.^15,20 One study included patients with DNR orders only (DNI orders were excluded).¹⁷ Three studies included patients with both DNR and DNI orders.^16,18,19 The numbers of enrolled patients with treatment limitations were generally low, with the two largest studies including 101 patients each on HFNC oxygen.^18,19

Risk of Bias

All included studies had a high risk of bias (Table 2). A high risk of bias was suggested because the investigations were single-center studies with unclear patient selection methods, did not explicitly report how decisions to limit treatments were made, and did not explicitly differentiate and separately analyze patients with “comfort measures only” goals of care.

Mortality

The hospital mortality rates of patients with DNI and/or DNR orders receiving HFNC were variable and ranged from 40% to 87%. In the two studies enrolling general ICU patient populations, the hospital mortality rates ranged from 40% to 60%. In the four studies enrolling patients with active malignancy, the hospital mortality rates ranged from 75% to 87%. No studies compared mortality rates with and without DNI and/or DNR orders.

Dyspnea, Work of Breathing, and Reduction in Opioid Doses

The impact of HFNC oxygen on symptom relief was reported in one retrospective observational study (published as a conference abstract only to date), which compared the effect of HFNC oxygen (n = 101) with conventional oxygen (n = 110).¹⁸ At first evaluation after hospital admission to a palliative care unit (after the patients had previously been started on either conventional oxygen or high-flow oxygen), patients in the HFNC oxygen group had worse (higher) dyspnea scores compared with patients who used conventional oxygen (Edmonton Symptom Assessment Scale score of 7.5 versus 5, P < .001). At follow-up, approximately 24 hours after admission to the hospital palliative care unit, there was no difference in the change of dyspnea between the HFNC oxygen group (dyspnea score change of 0) and the conventional oxygen group (dyspnea score change of −1, P = .18. In the same study, there was also no significant difference in the morphine dose requirement in each group, and exact doses were not reported.

Two studies reported improvement in oxygen saturation and respiratory rate after HFNC oxygen initiation (compared with before HFNC initiation).^16,20 Oxygen saturation increased from 89% to 95%, P < .01, in one study and 92% to 97%, P < .01, in a second study. The respiratory rate decreased from 31 to 25 breaths/minute in one study, and from 28 to 25 breaths/minute in a second study (both P < .01).

Quality of Life

No studies evaluated the quality of life of survivors.

Secondary Outcomes

Transition to Noninvasive Positive Pressure Ventilation

The proportion of patients who transitioned from HFNC oxygen to NPPV was relatively low in the two studies that reported this outcome, ranging from 0%²⁰ to 18%.¹⁶ In one observational study of a general ICU population, 9/50 (18%) of patients transitioned from HFNC oxygen to NPPV. There was no statistically significant difference in hospital mortality rates among those who progressed to NPPV (67%) versus those who did not progress to NPPV (58%), P = .72.

Tolerance of HFNC and Adverse Events

HFNC oxygen was generally well tolerated based on the assessment of three studies (Table 1). One study reported no adverse events,¹⁶ one study reported that HFNC oxygen had to be discontinued because of nasal discomfort in 1% of patients,¹⁹ and a second study reported that HFNC oxygen had to be discontinued because of agitation in 4% of patients.²⁰

Quality of Death in Nonsurvivors

No studies evaluated the quality of death in those patients who died.

In this systematic review of six studies, all with a high risk of bias, a significant proportion of patients with a DNI and/or DNR order who used HFNC oxygen survived to hospital discharge. Oxygen saturation and respiratory rate consistently improved in the three studies that reported these outcomes. Only one study (published as a conference abstract only to date),¹⁸ however, measured patient-important outcomes related to symptom management and found no significant difference in dyspnea or morphine dose requirements in patients on HFNC oxygen compared with patients on conventional oxygen. HFNC oxygen was generally well tolerated and only had to be stopped in <5% of patients due to intolerance. We found no studies that assessed the quality of life in survivors or the quality of death in nonsurvivors.

Based on the limited evidence in the included studies, HFNC may be a viable treatment option for patients with preset treatment limitations who have acute respiratory failure—with potential benefits of improved oxygenation, decreased respiratory rates, and hospital survival in a proportion of patients. Nevertheless, this systematic review highlights the vast paucity of data available to guide the use of HFNC oxygen in patients with treatment limitations and acute respiratory failure. Only a few studies, which were at high risk of bias, have been conducted on this topic to date. There is an inadequate evidence base to evaluate the comparative effectiveness of HFNC oxygen (versus NPPV versus conventional oxygen versus palliative opioids) in patients with DNI orders or comfort measures only orders.

Our review included two studies that evaluated the comparative effectiveness of HFNC oxygen in patients with DNI and/or DNR orders. The first retrospective observational study compared HFNC oxygen with conventional oxygen in patients with DNR and DNI orders and malignancy—and found no change in dyspnea—but did note an increase in mortality with HFNC oxygen (76% versus 51%).¹⁸ The second observational study compared HFNC oxygen with NPPV in patients with DNR orders with malignancy noted no difference in mortality.¹⁷ In patients with full-code orders, systematic reviews have shown that HFNC oxygen (compared with conventional oxygen) was associated with possible reductions in intubation rates, respiratory rates, and improvements in oxygenation—with no difference in mortality, dyspnea, patient comfort, or ICU/hospital length of stay. Compared with NPPV, HFNC oxygen was associated with similar rates of intubation and mortality.^4-6,21

Future studies in patients with acute respiratory failure and DNI and/or DNR orders should identify which treatment modality (HFNC oxygen compared with other modalities, such as NPPV, conventional oxygen, with or without palliative opioids) impacts outcomes, such as dyspnea reduction while maintaining an alert mental status, short- and long-term quality of life in survivors, and quality of death in nonsurvivors. Future studies should also identify the optimal treatment pathway to utilize when patients using HFNC oxygen fail this therapy (eg, transition to NPPV versus intensifying palliative opioids) as well as the optimal process to withdraw palliative HFNC oxygen.²² Identifying which patient populations may benefit from different treatment pathways should also be considered as different treatment strategies may be more beneficial in different patient populations (eg, based on cause and severity of acute respiratory failure). In addition, it should be noted that the primary goal of care might affect which outcomes are the most important to measure. While patients with comfort measures only, orders usually have a primary goal to prepare for a high-quality death, patients with DNI and/or DNR orders (but without comfort measures only orders) may have a primary goal to survive—but with the desire not to endure the high burden of intubation and mechanical ventilation if it became necessary. Finally, future studies should utilize high-quality study designs (eg, randomized controlled trials) that enable robust evaluation of comparative effectiveness of clinically relevant treatment strategies.

While several previous systematic reviews have evaluated the efficacy of HFNC in patients with acute respiratory failure without preset limitations on life support; to our knowledge, this is the first systematic review to assess outcomes in patients rigorously with preset treatment limitations. Our review is, however, limited by the high risk of bias of the studies that were included (single-center nature, retrospective observational study designs, small sample sizes, and lack of a description of how DNI and/or DNR statuses were determined) as well as the small number of studies available to be included.

This systematic review points to a significant evidence gap in our understanding of the role for HFNC oxygen (compared with other acceptable alternative treatment strategies) in adult patients with acute respiratory failure who have DNI and/or DNR orders. Further high-quality research is needed to explore these unanswered questions in an effort to best treat, guide, and engage in optimal end-of-life decision making among patients with acute respiratory failure.

High-flow nasal cannula (HFNC) oxygen therapy is effective in treating adults with acute hypoxemic respiratory failure, and to a lesser extent acute hypercapnic respiratory failure.^1-3 HFNC oxygen is capable of delivering oxygen with flows of 30-60 liters/minute, and can provide a high fraction of inspired oxygen, flush anatomic dead space, augment respiratory efforts, and provide mild continuous positive airway pressure effects. Several systematic reviews and meta-analyses have evaluated the effectiveness of HFNC oxygen and have shown modestly lower rates of intubation compared with conventional oxygen^4,5 and similar intubation rates compared with noninvasive positive pressure ventilation.^4-9 Although one randomized trial showed a lower risk of 90-day mortality for HFNC oxygen compared with either conventional oxygen or noninvasive positive pressure ventilation, several meta-analyses have shown no difference in intensive care unit (ICU) mortality.^4,6,8,10 The majority of studies have shown improvements in oxygenation, comfort, dyspnea scores, and breathing pattern with the initiation of HFNC oxygen.⁶

While the evidence to support the use of HFNC oxygen in patients with nonhypercapnic acute hypoxemic respiratory failure is growing, this evidence is based on patients enrolled in clinical trials who have no treatment limitations and consent to intubation if necessary. Indeed, several, if not all, randomized trials evaluating HFNC oxygen excluded patients who had do-not-intubate (DNI) or do-not-resuscitate (DNR) orders.^1,2,11 For patients with acute respiratory failure whose primary goal is not to extend life or utilize life support interventions such as invasive mechanical ventilation, HFNC oxygen may offer several benefits compared with other treatment options such as noninvasive positive pressure ventilation, conventional oxygen therapy, or palliative opioid therapy (Appendix Table 1). Determining which treatment options to use depends on the goals of care of the individual patient and the reasonable ability of a particular treatment to help the patient achieve those goals.

While a recent systematic review evaluated the existing evidence regarding the utility and outcomes of noninvasive positive pressure ventilation in adult patients with DNI orders,¹² a systematic review evaluating the evidence and rationale for HFNC oxygen in patients with DNI and/or DNR orders is lacking. Assessing such evidence is necessary to help clinicians and patients determine appropriate treatment choices and establish research priorities. Therefore, our primary objective was to determine what were the following outcomes: mortality, dyspnea, work of breathing, opioid doses, and quality of life in patients who received HFNC oxygen for acute respiratory failure and had a DNI and/or DNR order.

We conducted a systematic review of studies that evaluated patients who used HFNC oxygen for acute respiratory failure and had a DNI and/or DNR order. We reported the results using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statements.¹³ This review was registered with the PROSPERO registry, CRD42017059914.

We included studies that enrolled patients who were (1) hospitalized, (2) >18 years old, (3) had an acute respiratory failure of any cause, (4) received HFNC oxygen, and (5) had a DNI or DNR or comfort measures only order. We included publications of all study designs (interventional, observational, and posthoc analyses) and all languages. We excluded studies that enrolled <5 patients. If necessary, we contacted the authors of the included studies for additional information.

Our search strategy included the following databases from inception to October 14, 2018: PubMed, MEDLINE, CINAHL, MICROMEDEX, EMBASE, Web of Science, and Scopus. The database-specific search strategy was developed using an experienced librarian (Appendix Table 2). In addition, we screened the reference lists of systematic reviews as well as the included studies to find additional relevant articles. Two authors (AM, MEW) independently assessed the inclusion criteria of the titles and abstracts that were identified in the search. In addition, these two authors abstracted relevant data of the included studies.

Using the search strategy, we identified 2,757 citations and included 301 of these in the full-text review (Figure). We included six studies, which enrolled 293 patients in the final systematic review. Table 1 summarizes the characteristics of the included investigations, all of which were observational studies.^15-20 The studies were conducted in the United States of America (n = 3), Europe (n = 2), and Asia (n = 1). Two studies were conducted in the general ICU populations and included patients with hypoxemic respiratory failure only. Four studies were conducted in cancer populations in the hospital wards or ICU and did not specify the type of respiratory failure (hypoxemic versus hypercapnic). Two studies included patients with DNI orders only.^15,20 One study included patients with DNR orders only (DNI orders were excluded).¹⁷ Three studies included patients with both DNR and DNI orders.^16,18,19 The numbers of enrolled patients with treatment limitations were generally low, with the two largest studies including 101 patients each on HFNC oxygen.^18,19

Risk of Bias

All included studies had a high risk of bias (Table 2). A high risk of bias was suggested because the investigations were single-center studies with unclear patient selection methods, did not explicitly report how decisions to limit treatments were made, and did not explicitly differentiate and separately analyze patients with “comfort measures only” goals of care.

Mortality

The hospital mortality rates of patients with DNI and/or DNR orders receiving HFNC were variable and ranged from 40% to 87%. In the two studies enrolling general ICU patient populations, the hospital mortality rates ranged from 40% to 60%. In the four studies enrolling patients with active malignancy, the hospital mortality rates ranged from 75% to 87%. No studies compared mortality rates with and without DNI and/or DNR orders.

Dyspnea, Work of Breathing, and Reduction in Opioid Doses

The impact of HFNC oxygen on symptom relief was reported in one retrospective observational study (published as a conference abstract only to date), which compared the effect of HFNC oxygen (n = 101) with conventional oxygen (n = 110).¹⁸ At first evaluation after hospital admission to a palliative care unit (after the patients had previously been started on either conventional oxygen or high-flow oxygen), patients in the HFNC oxygen group had worse (higher) dyspnea scores compared with patients who used conventional oxygen (Edmonton Symptom Assessment Scale score of 7.5 versus 5, P < .001). At follow-up, approximately 24 hours after admission to the hospital palliative care unit, there was no difference in the change of dyspnea between the HFNC oxygen group (dyspnea score change of 0) and the conventional oxygen group (dyspnea score change of −1, P = .18. In the same study, there was also no significant difference in the morphine dose requirement in each group, and exact doses were not reported.

Two studies reported improvement in oxygen saturation and respiratory rate after HFNC oxygen initiation (compared with before HFNC initiation).^16,20 Oxygen saturation increased from 89% to 95%, P < .01, in one study and 92% to 97%, P < .01, in a second study. The respiratory rate decreased from 31 to 25 breaths/minute in one study, and from 28 to 25 breaths/minute in a second study (both P < .01).

Quality of Life

No studies evaluated the quality of life of survivors.

Secondary Outcomes

Transition to Noninvasive Positive Pressure Ventilation

The proportion of patients who transitioned from HFNC oxygen to NPPV was relatively low in the two studies that reported this outcome, ranging from 0%²⁰ to 18%.¹⁶ In one observational study of a general ICU population, 9/50 (18%) of patients transitioned from HFNC oxygen to NPPV. There was no statistically significant difference in hospital mortality rates among those who progressed to NPPV (67%) versus those who did not progress to NPPV (58%), P = .72.

Tolerance of HFNC and Adverse Events

HFNC oxygen was generally well tolerated based on the assessment of three studies (Table 1). One study reported no adverse events,¹⁶ one study reported that HFNC oxygen had to be discontinued because of nasal discomfort in 1% of patients,¹⁹ and a second study reported that HFNC oxygen had to be discontinued because of agitation in 4% of patients.²⁰

Quality of Death in Nonsurvivors

No studies evaluated the quality of death in those patients who died.

In this systematic review of six studies, all with a high risk of bias, a significant proportion of patients with a DNI and/or DNR order who used HFNC oxygen survived to hospital discharge. Oxygen saturation and respiratory rate consistently improved in the three studies that reported these outcomes. Only one study (published as a conference abstract only to date),¹⁸ however, measured patient-important outcomes related to symptom management and found no significant difference in dyspnea or morphine dose requirements in patients on HFNC oxygen compared with patients on conventional oxygen. HFNC oxygen was generally well tolerated and only had to be stopped in <5% of patients due to intolerance. We found no studies that assessed the quality of life in survivors or the quality of death in nonsurvivors.

Based on the limited evidence in the included studies, HFNC may be a viable treatment option for patients with preset treatment limitations who have acute respiratory failure—with potential benefits of improved oxygenation, decreased respiratory rates, and hospital survival in a proportion of patients. Nevertheless, this systematic review highlights the vast paucity of data available to guide the use of HFNC oxygen in patients with treatment limitations and acute respiratory failure. Only a few studies, which were at high risk of bias, have been conducted on this topic to date. There is an inadequate evidence base to evaluate the comparative effectiveness of HFNC oxygen (versus NPPV versus conventional oxygen versus palliative opioids) in patients with DNI orders or comfort measures only orders.

Our review included two studies that evaluated the comparative effectiveness of HFNC oxygen in patients with DNI and/or DNR orders. The first retrospective observational study compared HFNC oxygen with conventional oxygen in patients with DNR and DNI orders and malignancy—and found no change in dyspnea—but did note an increase in mortality with HFNC oxygen (76% versus 51%).¹⁸ The second observational study compared HFNC oxygen with NPPV in patients with DNR orders with malignancy noted no difference in mortality.¹⁷ In patients with full-code orders, systematic reviews have shown that HFNC oxygen (compared with conventional oxygen) was associated with possible reductions in intubation rates, respiratory rates, and improvements in oxygenation—with no difference in mortality, dyspnea, patient comfort, or ICU/hospital length of stay. Compared with NPPV, HFNC oxygen was associated with similar rates of intubation and mortality.^4-6,21

Future studies in patients with acute respiratory failure and DNI and/or DNR orders should identify which treatment modality (HFNC oxygen compared with other modalities, such as NPPV, conventional oxygen, with or without palliative opioids) impacts outcomes, such as dyspnea reduction while maintaining an alert mental status, short- and long-term quality of life in survivors, and quality of death in nonsurvivors. Future studies should also identify the optimal treatment pathway to utilize when patients using HFNC oxygen fail this therapy (eg, transition to NPPV versus intensifying palliative opioids) as well as the optimal process to withdraw palliative HFNC oxygen.²² Identifying which patient populations may benefit from different treatment pathways should also be considered as different treatment strategies may be more beneficial in different patient populations (eg, based on cause and severity of acute respiratory failure). In addition, it should be noted that the primary goal of care might affect which outcomes are the most important to measure. While patients with comfort measures only, orders usually have a primary goal to prepare for a high-quality death, patients with DNI and/or DNR orders (but without comfort measures only orders) may have a primary goal to survive—but with the desire not to endure the high burden of intubation and mechanical ventilation if it became necessary. Finally, future studies should utilize high-quality study designs (eg, randomized controlled trials) that enable robust evaluation of comparative effectiveness of clinically relevant treatment strategies.

While several previous systematic reviews have evaluated the efficacy of HFNC in patients with acute respiratory failure without preset limitations on life support; to our knowledge, this is the first systematic review to assess outcomes in patients rigorously with preset treatment limitations. Our review is, however, limited by the high risk of bias of the studies that were included (single-center nature, retrospective observational study designs, small sample sizes, and lack of a description of how DNI and/or DNR statuses were determined) as well as the small number of studies available to be included.

This systematic review points to a significant evidence gap in our understanding of the role for HFNC oxygen (compared with other acceptable alternative treatment strategies) in adult patients with acute respiratory failure who have DNI and/or DNR orders. Further high-quality research is needed to explore these unanswered questions in an effort to best treat, guide, and engage in optimal end-of-life decision making among patients with acute respiratory failure.

The recent designation of Pediatric Hospital Medicine (PHM) as a board-certified subspecialty has provided the opportunity to define which skills are core to hospitalist practice. One skill that is novel to the field and gaining traction is point-of-care ultrasonography (POCUS). POCUS differs from traditional ultrasonography in that it is performed at the bedside by the primary clinician and aims to answer a focused clinical question (eg, does this patient have a skin abscess?) rather than to provide a comprehensive evaluation of the anatomy and physiology. The proposed advantages of POCUS include real-time image interpretation, cost savings, procedural guidance to minimize complications, and reduction of ionizing radiation. Although specialties such as Critical Care (CC) and Emergency Medicine (EM) have integrated POCUS into their practice and training, best practices in PHM have not been defined. This Progress Note is a summary of recent evidence to update past reviews and set the stage for future PHM POCUS research and education.

We met with an academic librarian in March 2019 and performed a search of PubMed using Medical Subject Headings (MESH) terms associated with POCUS as well as Pediatrics. We limited our search to studies published within the past five years. The search was originally focused to the field of PHM before expanding to a broader search since very few studies were found that focused on Hospital Medicine or general pediatric ward populations. This initial search generated 274 publications. We then performed a supplemental literature search using references from studies found in our initial search, as well as further ad hoc searching in Embase and Google Scholar.

After our literature search, we reviewed the PHM core competencies and identified the common clinical diagnoses and core skills for which there is POCUS literature published in the past five years. These included acute abdominal pain, bronchiolitis, pneumonia, skin and soft tissue infection, newborn care/delivery room management, bladder catheterization, fluid management, intravenous access, and lumbar puncture (LP). We chose to focus on one skill and two diagnoses that were generalizable to pediatric hospitalists across different settings and for which there was compelling evidence for POCUS use, such as pneumonia, skin abscess, and LP. We found few studies that included general pediatric ward patients, but we considered EM and CC studies to be relevant as several pediatric hospitalists practice in these clinical settings and with these patient populations.

POCUS can be useful for diagnosing pneumonia by direct visualization of lung consolidation or by identification of various sonographic artifacts that suggest pathology. For example, “B-lines” are vertical artifacts that extend from the pleura and suggest interstitial fluid or pneumonia when they are present in abnormally high numbers or density. POCUS can also be used to diagnose parapneumonic effusions by scanning dependent areas of the lung (eg, the diaphragm in children sitting upright) and looking for anechoic or hypoechoic areas.

Three recent meta-analyses found favorable operating characteristics when using POCUS for the diagnosis of pneumonia in children, with summary sensitivities of 93%-94% and specificities of 92%-96%.^1-3 However, these meta-analyses were limited by high heterogeneity due to the inclusion of multiple different care settings and the use of variable reference standards and sonographic criteria for diagnosing pneumonia. POCUS is superior to chest radiography for evaluating parapneumonic pleural effusions,⁴ allowing for rapid identification of loculations, fibrin strands, and proteinaceous material, and for serial bedside evaluation of effusion size and characteristics.

Additional advantages of POCUS include avoidance of ionizing radiation and the potential for cost and time savings. Two studies demonstrated reductions in radiography use and improved cost, although they were not conducted on hospitalized patients. One randomized controlled trial (RCT) conducted in a pediatric emergency department (ED) demonstrated a 38.8% reduction in chest radiography use without increasing the ED length of stay (EDLOS), antibiotic use, or unscheduled follow-up visits.⁵ A retrospective matched cohort study conducted in another pediatric ED reported that when compared with patients evaluated by chest radiography, those evaluated by POCUS had significantly shorter EDLOS (−60.9 min) and mean health systems savings ($187 per patient).⁶ We believe that POCUS has value in the evaluation and management of pneumonia and parapneumonic effusions, although further studies investigating patient outcomes and involving inpatient populations are required.

POCUS can augment the physical examination, helping to both avoid unnecessary incision and drainage (I+D) procedures and detect drainable fluid collections. Abscess is suggested when hypoechoic material without vascular flow is detected, and although other structures such as vessels, cysts, and lymph nodes can mimic skin abscesses, this is a relatively straightforward examination for clinicians to learn.

Two meta-analyses found that POCUS had high sensitivity for diagnosing skin abscesses in the ED.^7,8 A pediatric subgroup analysis conducted in a study by Barbic et al. found a sensitivity and a specificity of 94% (95% C: 88%-98%) and 83% (95% C: 47%-97%), respectively.⁷ Subramaniam et al. included six studies (four pediatric) with 800 patients (653 ≤ 18 years old) and found an overall pooled sensitivity of 97% (95% C: 94%-98%) and a specificity of 83% (95% C: 75%-88%).⁸ No subgroup analysis was performed, but the included pediatric studies reported sensitivities and specificities between 90%-98% and 68%-87%, respectively.

Although POCUS performs better than physical examination for the diagnosis of drainable abscesses, evidence regarding patient outcomes is mixed. A retrospective review from four pediatric EDs found that integration of POCUS lowered treatment failure rates, defined as any incision and drainage (I+D) or surgical manipulation after discharge from the initial ED visit (4.4% vs 15.6%, P < .005).⁹ A single-center retrospective cohort study found that POCUS reduced EDLOS by a median of 73 minutes (95% C: 52-94 min) when compared with radiology-performed studies.¹⁰ The aforementioned study conducted by Barbic et al. found that in pediatric studies, POCUS led to a change in management (eg, whether or not to attempt I+D) in 14%-27% of patients.⁷ However, a multicenter prospective observational cohort study involving seven pediatric EDs found that despite changing the management in 22.9% of cases, POCUS was not associated with any statistically significant differences in treatment failure rates, EDLOS, discharge rates, use of sedation, or use of alternative imaging.¹¹ These studies were limited by a lack of randomization or control group and emphasize the need for RCTs that measure patient outcomes. Future studies should investigate how POCUS can be used in inpatient settings both for initial diagnosis of drainable abscesses and for serial evaluation of evolving phlegmon or incompletely drained collections.

LP is commonly performed by pediatric hospitalists, although success can be influenced by numerous factors, including provider and staff expertise, patient anatomy, and body habitus. Requiring multiple attempts can increase patient discomfort and parental anxiety. Failure to obtain cerebrospinal fluid can delay diagnosis or leave providers in uncertain clinical situations that may commit patients to prolonged antibiotic courses. POCUS can be used to identify anatomic markers such as interspinous processes, anatomic midline, and depth of the ligamentum flavum.¹² It can also be used to identify epidural hematomas after failed LPs to avoid additional unsuccessful attempts.¹³ POCUS guidance for LP has been described using both static (preprocedural marking) and dynamic (scanning during the procedure) techniques, although most of the studies use the static approach. The Society for Hospital Medicine POCUS Task Force has recently released a position statement recommending that POCUS should be used for site selection before performing LP in adult patients when providers are adequately trained.¹² Although this position statement was for adult patients, recent evidence suggests that there is also benefit in Pediatrics.

Two recent meta-analyses have investigated POCUS use for pediatric LPs.^14,15 Olowoyeye et al. included four studies with a total of 277 patients and found that POCUS use was associated with a reduction in traumatic taps (risk ratio [RR] = 0.53, 95% C: 0.13-0.82) when compared with landmark approaches.¹⁴ However, there was no statistically significant reduction in LP failure, number of needle insertion attempts, or procedure length. A more recent meta-analysis performed a pediatric subgroup analysis of six studies including 452 patients and found a statistically significant reduction in traumatic taps (13.7% vs 31.8%, risk difference = −21.3%, 95% C: −38.2% to −4.3%) and number of needle insertion attempts (1.53 vs 2.07, mean difference = −0.47, 95% C: −0.73 to −0.21).¹⁵ The primary outcome of LP success trended toward favoring POCUS, but it was not statistically significant (88.4% vs 74.0%, OR = 2.55, 95% C: 0.99-6.52). We believe that recent evidence suggests that there is benefit in using POCUS when hospitalists attempt pediatric LPs, particularly when physical landmarks are difficult to identify or after failed attempts. However, adequate training with simulation and supervised practice should be undertaken before integrating this into clinical practice.

Evidence accumulated in the past five years has built on previous work suggesting that POCUS has a role in the diagnosis of pneumonia and skin abscess and in the performance of LPs. However, gaps in the literature remain when applying POCUS in PHM. Only a few studies to date were conducted in non-CC inpatient settings, and although several pediatric hospitalists work in EDs or care for critically ill children, our largest population comprises general pediatric ward patients. Studies have also used ultrasonographers with variable POCUS training and clinical experience, which makes comparing or combining studies challenging since POCUS is dependent on provider skills. Studies involving PHM providers and inpatient populations are needed. Additional studies evaluating the process and outcome measures are also needed to understand whether the theoretical advantages are consistently realized in real-world PHM practice. Finally, PHM-specific curricula should be designed in collaboration with various PHM stakeholders and with specialties who already have robust POCUS training pathways. There is opportunity within PHM for multi institutional research collaboration, identification of best practices, and development of PHM-specific training for fellowship and faculty development programs.

The recent designation of Pediatric Hospital Medicine (PHM) as a board-certified subspecialty has provided the opportunity to define which skills are core to hospitalist practice. One skill that is novel to the field and gaining traction is point-of-care ultrasonography (POCUS). POCUS differs from traditional ultrasonography in that it is performed at the bedside by the primary clinician and aims to answer a focused clinical question (eg, does this patient have a skin abscess?) rather than to provide a comprehensive evaluation of the anatomy and physiology. The proposed advantages of POCUS include real-time image interpretation, cost savings, procedural guidance to minimize complications, and reduction of ionizing radiation. Although specialties such as Critical Care (CC) and Emergency Medicine (EM) have integrated POCUS into their practice and training, best practices in PHM have not been defined. This Progress Note is a summary of recent evidence to update past reviews and set the stage for future PHM POCUS research and education.

We met with an academic librarian in March 2019 and performed a search of PubMed using Medical Subject Headings (MESH) terms associated with POCUS as well as Pediatrics. We limited our search to studies published within the past five years. The search was originally focused to the field of PHM before expanding to a broader search since very few studies were found that focused on Hospital Medicine or general pediatric ward populations. This initial search generated 274 publications. We then performed a supplemental literature search using references from studies found in our initial search, as well as further ad hoc searching in Embase and Google Scholar.

After our literature search, we reviewed the PHM core competencies and identified the common clinical diagnoses and core skills for which there is POCUS literature published in the past five years. These included acute abdominal pain, bronchiolitis, pneumonia, skin and soft tissue infection, newborn care/delivery room management, bladder catheterization, fluid management, intravenous access, and lumbar puncture (LP). We chose to focus on one skill and two diagnoses that were generalizable to pediatric hospitalists across different settings and for which there was compelling evidence for POCUS use, such as pneumonia, skin abscess, and LP. We found few studies that included general pediatric ward patients, but we considered EM and CC studies to be relevant as several pediatric hospitalists practice in these clinical settings and with these patient populations.

POCUS can be useful for diagnosing pneumonia by direct visualization of lung consolidation or by identification of various sonographic artifacts that suggest pathology. For example, “B-lines” are vertical artifacts that extend from the pleura and suggest interstitial fluid or pneumonia when they are present in abnormally high numbers or density. POCUS can also be used to diagnose parapneumonic effusions by scanning dependent areas of the lung (eg, the diaphragm in children sitting upright) and looking for anechoic or hypoechoic areas.

Three recent meta-analyses found favorable operating characteristics when using POCUS for the diagnosis of pneumonia in children, with summary sensitivities of 93%-94% and specificities of 92%-96%.^1-3 However, these meta-analyses were limited by high heterogeneity due to the inclusion of multiple different care settings and the use of variable reference standards and sonographic criteria for diagnosing pneumonia. POCUS is superior to chest radiography for evaluating parapneumonic pleural effusions,⁴ allowing for rapid identification of loculations, fibrin strands, and proteinaceous material, and for serial bedside evaluation of effusion size and characteristics.

Additional advantages of POCUS include avoidance of ionizing radiation and the potential for cost and time savings. Two studies demonstrated reductions in radiography use and improved cost, although they were not conducted on hospitalized patients. One randomized controlled trial (RCT) conducted in a pediatric emergency department (ED) demonstrated a 38.8% reduction in chest radiography use without increasing the ED length of stay (EDLOS), antibiotic use, or unscheduled follow-up visits.⁵ A retrospective matched cohort study conducted in another pediatric ED reported that when compared with patients evaluated by chest radiography, those evaluated by POCUS had significantly shorter EDLOS (−60.9 min) and mean health systems savings ($187 per patient).⁶ We believe that POCUS has value in the evaluation and management of pneumonia and parapneumonic effusions, although further studies investigating patient outcomes and involving inpatient populations are required.

POCUS can augment the physical examination, helping to both avoid unnecessary incision and drainage (I+D) procedures and detect drainable fluid collections. Abscess is suggested when hypoechoic material without vascular flow is detected, and although other structures such as vessels, cysts, and lymph nodes can mimic skin abscesses, this is a relatively straightforward examination for clinicians to learn.

Two meta-analyses found that POCUS had high sensitivity for diagnosing skin abscesses in the ED.^7,8 A pediatric subgroup analysis conducted in a study by Barbic et al. found a sensitivity and a specificity of 94% (95% C: 88%-98%) and 83% (95% C: 47%-97%), respectively.⁷ Subramaniam et al. included six studies (four pediatric) with 800 patients (653 ≤ 18 years old) and found an overall pooled sensitivity of 97% (95% C: 94%-98%) and a specificity of 83% (95% C: 75%-88%).⁸ No subgroup analysis was performed, but the included pediatric studies reported sensitivities and specificities between 90%-98% and 68%-87%, respectively.

Although POCUS performs better than physical examination for the diagnosis of drainable abscesses, evidence regarding patient outcomes is mixed. A retrospective review from four pediatric EDs found that integration of POCUS lowered treatment failure rates, defined as any incision and drainage (I+D) or surgical manipulation after discharge from the initial ED visit (4.4% vs 15.6%, P < .005).⁹ A single-center retrospective cohort study found that POCUS reduced EDLOS by a median of 73 minutes (95% C: 52-94 min) when compared with radiology-performed studies.¹⁰ The aforementioned study conducted by Barbic et al. found that in pediatric studies, POCUS led to a change in management (eg, whether or not to attempt I+D) in 14%-27% of patients.⁷ However, a multicenter prospective observational cohort study involving seven pediatric EDs found that despite changing the management in 22.9% of cases, POCUS was not associated with any statistically significant differences in treatment failure rates, EDLOS, discharge rates, use of sedation, or use of alternative imaging.¹¹ These studies were limited by a lack of randomization or control group and emphasize the need for RCTs that measure patient outcomes. Future studies should investigate how POCUS can be used in inpatient settings both for initial diagnosis of drainable abscesses and for serial evaluation of evolving phlegmon or incompletely drained collections.

LP is commonly performed by pediatric hospitalists, although success can be influenced by numerous factors, including provider and staff expertise, patient anatomy, and body habitus. Requiring multiple attempts can increase patient discomfort and parental anxiety. Failure to obtain cerebrospinal fluid can delay diagnosis or leave providers in uncertain clinical situations that may commit patients to prolonged antibiotic courses. POCUS can be used to identify anatomic markers such as interspinous processes, anatomic midline, and depth of the ligamentum flavum.¹² It can also be used to identify epidural hematomas after failed LPs to avoid additional unsuccessful attempts.¹³ POCUS guidance for LP has been described using both static (preprocedural marking) and dynamic (scanning during the procedure) techniques, although most of the studies use the static approach. The Society for Hospital Medicine POCUS Task Force has recently released a position statement recommending that POCUS should be used for site selection before performing LP in adult patients when providers are adequately trained.¹² Although this position statement was for adult patients, recent evidence suggests that there is also benefit in Pediatrics.

Two recent meta-analyses have investigated POCUS use for pediatric LPs.^14,15 Olowoyeye et al. included four studies with a total of 277 patients and found that POCUS use was associated with a reduction in traumatic taps (risk ratio [RR] = 0.53, 95% C: 0.13-0.82) when compared with landmark approaches.¹⁴ However, there was no statistically significant reduction in LP failure, number of needle insertion attempts, or procedure length. A more recent meta-analysis performed a pediatric subgroup analysis of six studies including 452 patients and found a statistically significant reduction in traumatic taps (13.7% vs 31.8%, risk difference = −21.3%, 95% C: −38.2% to −4.3%) and number of needle insertion attempts (1.53 vs 2.07, mean difference = −0.47, 95% C: −0.73 to −0.21).¹⁵ The primary outcome of LP success trended toward favoring POCUS, but it was not statistically significant (88.4% vs 74.0%, OR = 2.55, 95% C: 0.99-6.52). We believe that recent evidence suggests that there is benefit in using POCUS when hospitalists attempt pediatric LPs, particularly when physical landmarks are difficult to identify or after failed attempts. However, adequate training with simulation and supervised practice should be undertaken before integrating this into clinical practice.

Evidence accumulated in the past five years has built on previous work suggesting that POCUS has a role in the diagnosis of pneumonia and skin abscess and in the performance of LPs. However, gaps in the literature remain when applying POCUS in PHM. Only a few studies to date were conducted in non-CC inpatient settings, and although several pediatric hospitalists work in EDs or care for critically ill children, our largest population comprises general pediatric ward patients. Studies have also used ultrasonographers with variable POCUS training and clinical experience, which makes comparing or combining studies challenging since POCUS is dependent on provider skills. Studies involving PHM providers and inpatient populations are needed. Additional studies evaluating the process and outcome measures are also needed to understand whether the theoretical advantages are consistently realized in real-world PHM practice. Finally, PHM-specific curricula should be designed in collaboration with various PHM stakeholders and with specialties who already have robust POCUS training pathways. There is opportunity within PHM for multi institutional research collaboration, identification of best practices, and development of PHM-specific training for fellowship and faculty development programs.

The recent designation of Pediatric Hospital Medicine (PHM) as a board-certified subspecialty has provided the opportunity to define which skills are core to hospitalist practice. One skill that is novel to the field and gaining traction is point-of-care ultrasonography (POCUS). POCUS differs from traditional ultrasonography in that it is performed at the bedside by the primary clinician and aims to answer a focused clinical question (eg, does this patient have a skin abscess?) rather than to provide a comprehensive evaluation of the anatomy and physiology. The proposed advantages of POCUS include real-time image interpretation, cost savings, procedural guidance to minimize complications, and reduction of ionizing radiation. Although specialties such as Critical Care (CC) and Emergency Medicine (EM) have integrated POCUS into their practice and training, best practices in PHM have not been defined. This Progress Note is a summary of recent evidence to update past reviews and set the stage for future PHM POCUS research and education.

We met with an academic librarian in March 2019 and performed a search of PubMed using Medical Subject Headings (MESH) terms associated with POCUS as well as Pediatrics. We limited our search to studies published within the past five years. The search was originally focused to the field of PHM before expanding to a broader search since very few studies were found that focused on Hospital Medicine or general pediatric ward populations. This initial search generated 274 publications. We then performed a supplemental literature search using references from studies found in our initial search, as well as further ad hoc searching in Embase and Google Scholar.

After our literature search, we reviewed the PHM core competencies and identified the common clinical diagnoses and core skills for which there is POCUS literature published in the past five years. These included acute abdominal pain, bronchiolitis, pneumonia, skin and soft tissue infection, newborn care/delivery room management, bladder catheterization, fluid management, intravenous access, and lumbar puncture (LP). We chose to focus on one skill and two diagnoses that were generalizable to pediatric hospitalists across different settings and for which there was compelling evidence for POCUS use, such as pneumonia, skin abscess, and LP. We found few studies that included general pediatric ward patients, but we considered EM and CC studies to be relevant as several pediatric hospitalists practice in these clinical settings and with these patient populations.

POCUS can be useful for diagnosing pneumonia by direct visualization of lung consolidation or by identification of various sonographic artifacts that suggest pathology. For example, “B-lines” are vertical artifacts that extend from the pleura and suggest interstitial fluid or pneumonia when they are present in abnormally high numbers or density. POCUS can also be used to diagnose parapneumonic effusions by scanning dependent areas of the lung (eg, the diaphragm in children sitting upright) and looking for anechoic or hypoechoic areas.

Three recent meta-analyses found favorable operating characteristics when using POCUS for the diagnosis of pneumonia in children, with summary sensitivities of 93%-94% and specificities of 92%-96%.^1-3 However, these meta-analyses were limited by high heterogeneity due to the inclusion of multiple different care settings and the use of variable reference standards and sonographic criteria for diagnosing pneumonia. POCUS is superior to chest radiography for evaluating parapneumonic pleural effusions,⁴ allowing for rapid identification of loculations, fibrin strands, and proteinaceous material, and for serial bedside evaluation of effusion size and characteristics.

Additional advantages of POCUS include avoidance of ionizing radiation and the potential for cost and time savings. Two studies demonstrated reductions in radiography use and improved cost, although they were not conducted on hospitalized patients. One randomized controlled trial (RCT) conducted in a pediatric emergency department (ED) demonstrated a 38.8% reduction in chest radiography use without increasing the ED length of stay (EDLOS), antibiotic use, or unscheduled follow-up visits.⁵ A retrospective matched cohort study conducted in another pediatric ED reported that when compared with patients evaluated by chest radiography, those evaluated by POCUS had significantly shorter EDLOS (−60.9 min) and mean health systems savings ($187 per patient).⁶ We believe that POCUS has value in the evaluation and management of pneumonia and parapneumonic effusions, although further studies investigating patient outcomes and involving inpatient populations are required.

POCUS can augment the physical examination, helping to both avoid unnecessary incision and drainage (I+D) procedures and detect drainable fluid collections. Abscess is suggested when hypoechoic material without vascular flow is detected, and although other structures such as vessels, cysts, and lymph nodes can mimic skin abscesses, this is a relatively straightforward examination for clinicians to learn.

Two meta-analyses found that POCUS had high sensitivity for diagnosing skin abscesses in the ED.^7,8 A pediatric subgroup analysis conducted in a study by Barbic et al. found a sensitivity and a specificity of 94% (95% C: 88%-98%) and 83% (95% C: 47%-97%), respectively.⁷ Subramaniam et al. included six studies (four pediatric) with 800 patients (653 ≤ 18 years old) and found an overall pooled sensitivity of 97% (95% C: 94%-98%) and a specificity of 83% (95% C: 75%-88%).⁸ No subgroup analysis was performed, but the included pediatric studies reported sensitivities and specificities between 90%-98% and 68%-87%, respectively.

Although POCUS performs better than physical examination for the diagnosis of drainable abscesses, evidence regarding patient outcomes is mixed. A retrospective review from four pediatric EDs found that integration of POCUS lowered treatment failure rates, defined as any incision and drainage (I+D) or surgical manipulation after discharge from the initial ED visit (4.4% vs 15.6%, P < .005).⁹ A single-center retrospective cohort study found that POCUS reduced EDLOS by a median of 73 minutes (95% C: 52-94 min) when compared with radiology-performed studies.¹⁰ The aforementioned study conducted by Barbic et al. found that in pediatric studies, POCUS led to a change in management (eg, whether or not to attempt I+D) in 14%-27% of patients.⁷ However, a multicenter prospective observational cohort study involving seven pediatric EDs found that despite changing the management in 22.9% of cases, POCUS was not associated with any statistically significant differences in treatment failure rates, EDLOS, discharge rates, use of sedation, or use of alternative imaging.¹¹ These studies were limited by a lack of randomization or control group and emphasize the need for RCTs that measure patient outcomes. Future studies should investigate how POCUS can be used in inpatient settings both for initial diagnosis of drainable abscesses and for serial evaluation of evolving phlegmon or incompletely drained collections.

LP is commonly performed by pediatric hospitalists, although success can be influenced by numerous factors, including provider and staff expertise, patient anatomy, and body habitus. Requiring multiple attempts can increase patient discomfort and parental anxiety. Failure to obtain cerebrospinal fluid can delay diagnosis or leave providers in uncertain clinical situations that may commit patients to prolonged antibiotic courses. POCUS can be used to identify anatomic markers such as interspinous processes, anatomic midline, and depth of the ligamentum flavum.¹² It can also be used to identify epidural hematomas after failed LPs to avoid additional unsuccessful attempts.¹³ POCUS guidance for LP has been described using both static (preprocedural marking) and dynamic (scanning during the procedure) techniques, although most of the studies use the static approach. The Society for Hospital Medicine POCUS Task Force has recently released a position statement recommending that POCUS should be used for site selection before performing LP in adult patients when providers are adequately trained.¹² Although this position statement was for adult patients, recent evidence suggests that there is also benefit in Pediatrics.

Two recent meta-analyses have investigated POCUS use for pediatric LPs.^14,15 Olowoyeye et al. included four studies with a total of 277 patients and found that POCUS use was associated with a reduction in traumatic taps (risk ratio [RR] = 0.53, 95% C: 0.13-0.82) when compared with landmark approaches.¹⁴ However, there was no statistically significant reduction in LP failure, number of needle insertion attempts, or procedure length. A more recent meta-analysis performed a pediatric subgroup analysis of six studies including 452 patients and found a statistically significant reduction in traumatic taps (13.7% vs 31.8%, risk difference = −21.3%, 95% C: −38.2% to −4.3%) and number of needle insertion attempts (1.53 vs 2.07, mean difference = −0.47, 95% C: −0.73 to −0.21).¹⁵ The primary outcome of LP success trended toward favoring POCUS, but it was not statistically significant (88.4% vs 74.0%, OR = 2.55, 95% C: 0.99-6.52). We believe that recent evidence suggests that there is benefit in using POCUS when hospitalists attempt pediatric LPs, particularly when physical landmarks are difficult to identify or after failed attempts. However, adequate training with simulation and supervised practice should be undertaken before integrating this into clinical practice.

Evidence accumulated in the past five years has built on previous work suggesting that POCUS has a role in the diagnosis of pneumonia and skin abscess and in the performance of LPs. However, gaps in the literature remain when applying POCUS in PHM. Only a few studies to date were conducted in non-CC inpatient settings, and although several pediatric hospitalists work in EDs or care for critically ill children, our largest population comprises general pediatric ward patients. Studies have also used ultrasonographers with variable POCUS training and clinical experience, which makes comparing or combining studies challenging since POCUS is dependent on provider skills. Studies involving PHM providers and inpatient populations are needed. Additional studies evaluating the process and outcome measures are also needed to understand whether the theoretical advantages are consistently realized in real-world PHM practice. Finally, PHM-specific curricula should be designed in collaboration with various PHM stakeholders and with specialties who already have robust POCUS training pathways. There is opportunity within PHM for multi institutional research collaboration, identification of best practices, and development of PHM-specific training for fellowship and faculty development programs.

Research, in the field of Hospital Medicine, often leverages data collected for reasons other than research. For example, electronic medical record data or patient satisfaction survey results can be used to answer questions that are relevant to the practice of hospital medicine. In these types of datasets, data will inevitably be missing. Such missing data can compromise our ability to draw definitive conclusions from our research study. This review introduces the concept of missing data, describes patterns and mechanisms of missing data, and discusses common approaches for the handling of missing data, including sensitivity analyses for determining how robust the results are despite assumptions made about the missing data.

Missing data create a host of problems for researchers. First, missing data result in a loss of information and can diminish the power of the proposed study. Second, the irregular data complicate the analysis because many of the standard software procedures used have been developed for fully observed or “complete” data (ie, each subject has a value for all measures of interest). Finally, missing data may introduce bias due to the systematic difference between the observed and the unobserved data. For example, if men are less likely than women to complete all questions in a patient satisfaction survey when they are not satisfied, then hospital satisfaction analyses that rely on completed surveys would tend to provide biased estimates of the satisfaction males have with their care.

The ideal approach to mitigating problems caused by missing data is to anticipate and incorporate strategies to minimize missing data into the study design (ie, when planning data collection protocols for prospective studies). This plan should provide strategies for minimizing nonresponse and estimating the magnitude of anticipated missing data to ensure that the study achieves sufficient strength despite the missing data.

Strategies for minimizing nonresponse include (1) informing potential study participants, at initial contact, about the implications of missing data on the ability to answer the research question; (2) collecting several phone numbers, addresses, preferred method of contact and calling times, as well as an alternative contact, in case the primary study contact is unable to be reached; (3) specifying the number of call backs, as well as the time of contact; and (4) piloting data capture questions for phrasing, clarity, and sensitivity, in order to resolve problems before initiating the study. One approach that can be used to mitigate the impact of missing data in surveys is to contact a sample of the initial nonrespondents using a more intensive follow-up approach (eg, a nonresponse to a mailed survey is followed up by a telephone call in order to conduct the survey again over the phone), and this is referred to as “nonresponse two-phase sampling.” The additional data, captured in the second phase, not only reduces the nonresponse rate but can also provide important information on the missing data mechanism.^1,2 In longitudinal studies with dropouts, one can measure participants’ intent to drop out in order to evaluate how much the probability of dropping out depends on missing responses.³ One may also choose to determine the power and implications of sample size under different missing data assumptions.⁴

Different data sources are likely to have unique reasons for missing values due to the workflows involved in how the data are collected. In research involving the use of data from electronic medical records, missing data on specific diagnoses involving patients who are regularly engaged in care are often considered to be “not present” or “normal”, since clinical documentation workflows are largely governed by the concept of “documentation by exception” in which diagnoses are documented only when there is an exception to the expectation that these are not present. For example, “diabetes mellitus” is commonly documented, but “diabetes mellitus not present” is rarely documented in electronic medical records which are used for clinical care. Thus, lack of explicit documentation is likely to indicate that diabetes mellitus is, in fact, not present.

Certain variables may be missing simply because there is no quantifiable value­—ie, the data do not exist. Structural missingness refers to a value that does not exist for a logical reason (eg, “What is the gender of your first child?” for those who do not have a child). Censoring, which occurs during “time to event” analysis, refers to a situation where information about a subject stops before the event of interest happens, for example, when a subject in a study involving a 30-day outcome dies at day 14. The term “limit of detection” refers to the lowest or highest level at which two distinct values can reasonably be distinguished (eg, the lower limit of detection of a C-reactive protein assay may be 1 mg/dL, so lower values might simply be reported by the lab as <1 mg/dL).⁵ These types of missing data require specific methods that are not discussed in this review.

These examples illustrate that approaches to dealing with missing data vary depending on what data sources are used and how data are collected. Understanding the reasons missing data are present is a necessary step in formulating a robust analytic approach to handling missing data.

Missing Data Patterns

Evaluating missing data patterns provides information on the degree and complexity of the missing data problem and can aid in choosing an appropriate missing data handling method. This is because some analytic methods work well for a general pattern (nonmonotone) and other methods work for special patterns (eg, monotone, file matching). In longitudinal studies, missing data is commonly missing in a monotone pattern, where once one variable is missing then all subsequent variables are also missing for a particular subject. This occurs when a study participant is lost to follow-up. For example, a monotone missing data pattern may occur in a study that requires a series of follow-up visits for laboratory blood tests. If a patient drops out, it results in a monotone missing data pattern, as no data on blood test results are available once the patient drops out. If the patient just skips an intermediate visit but returns for the final blood test, this would show a nonmonotone missing data pattern. A file-matching pattern occurs when variables are never observed together. This pattern can occur when data from several studies are merged and some variables are not collected in all studies. For example, three studies are merged and all three collect blood pressure, but only one study collects age and only one study collects sex.

Missing Data Mechanisms

The missing data mechanism relates to the underlying reasons for missing values and the relationships between variables with and without missing data. In general, missing data can be either random or nonrandom with distinctions in randomness made by three types: (1) data missing completely at random (MCAR); (2) data missing at random (MAR); and (3) data missing not at random (MNAR).⁶ As with the missing data pattern, understanding the missing data mechanism can aid in selecting an appropriate approach to handling the missing data.

Data are MCAR if the missingness does not depend on any study variables, meaning that all subjects are equally likely to be missing certain data elements. When the data are MCAR, those with missing values can be viewed as a simple random sample from the complete (but never actually observed) data and can be dropped from analysis without causing bias in the results. If the values of some diagnostic tests were missing for some patients due to equipment malfunction or electricity outage, for example, then the missingness may be considered MCAR.

Data are MAR if the missingness depends on the observed characteristics but not the unobserved characteristics, meaning that the relationships observed in the data can be used to predict the occurrence of missing values. Because the “randomness” of MAR is conditional on observed characteristics, which distinguishes it from the “completely at random” type of MCAR, dropping or omitting those cases with missing values from the analysis may lead to biased results.⁷ In a study of quality of life (QOL) for patients with mild to moderate traumatic brain injury, if health-related QOL questions were not answered by some patients with high pain levels (even though the pain levels were recorded), the missingness of QOL may be considered as MAR. This is due to the fact that within subjects grouped by the observed characteristic of pain (that is, conditional on similar levels of pain) the missingness of QOL is the result of chance and does not depend on the values (observed or unobserved) of QOL. It follows then, that once grouped into a high (or low) pain stratum, if QOL is considered MAR, then, whether or not it is observed, is random.

Data are considered MNAR if their missingness depends on characteristics that are not observed and cannot be fully explained by the observed characteristics. Systematic differences between missing and nonmissing data exist for data that is MNAR. For example, if a survey of household income had an increased probability of missing incomes from the low-income families then the data would be considered as MNAR.

Randomness in the missing data mechanism may be ignored without affecting the inference in some circumstances.⁸ Both MCAR and MAR can be considered as “ignorable” in the sense that a proper method (eg, multiple imputation) may recover the missing information without modeling (ie, accounting for) the random process of the missing data mechanism (Table).⁹ In contrast, the MNAR mechanism requires a method that takes into account the missing data mechanism in order to make inferences about the complete (and partially unobserved) data; or in other words, a model for the missing data mechanism cannot be ignored. It is for this reason that the MNAR mechanism is often called “nonignorable”. Nonignorable missing data present a challenge to researchers because the mechanism underlying the missingness must be included in the analysis. Yet researchers rarely know what the missingness mechanism is, and the data needed to validate any putative mechanism is, in fact, missing. In cases when more than one variable is subject to missingness, researchers need to assess the missingness mechanism for each variable and tailor their approach to the specific missing data problems.⁹

There is no universally accepted standard to guide when statistical methods should be applied to account for missing data. The amount of missing data alone cannot fully assess the missing data problem; missing data patterns and mechanisms can have greater impact on research results than the proportion of missing data alone. A good statistical method for handling missing data should provide an unbiased estimate of the quantity that the investigators intend to estimate; make use of the partial information in the incomplete cases to improve efficiency (and in most cases also to reduce bias); and provide valid estimates of the standard errors, confidence intervals, and P values for statistical tests. There are generally four broadly defined classes of methods for handling missing data in clinical research: (1) the complete-case analysis, (2) single imputation methods, (3) the weighted estimating-equation approach, and (4) the model-based approach including maximum likelihood (ML) and multiple imputation (Table and Appendix).¹⁰

Since missing data mechanisms cannot be conclusively verified, it is good practice to conduct some sensitivity analyses to test the robustness of the primary results. For this purpose, pattern-mixture models provide a flexible framework for implementing sensitivity analyses to missing data assumptions and can be used to evaluate the possibility of the data being MNAR. In this framework, the missing data distribution is modeled and then incorporated into the outcome model of interest. Tipping-point analysis is a sensitivity analysis where the missing data is replaced with a range of values to determine how much the values must change for the results of the study to tip from significant to not significant. If the same general conclusions remain valid over a range of assumptions about the missing data values, then one can have greater confidence in the study conclusions.

In dealing with missing data from clinical research, clinicians and statisticians need to work together to minimize missingness at the data collection stage, document the reasons for missingness, use substantive knowledge, if possible, to assess the missing data mechanism, perform primary analysis based on a defensible missing data mechanism, and conduct a sensitivity analysis to assess whether the primary result is robust despite departure from the assumed missing data mechanism.

Acknowledgments

The following members of the Journal of Hospital Medicine Leadership team contributed to this review: Mel L. Anderson, MD; Peter Cram, MD, MBA; JoAnna K. Leyenaar, MD, PhD, MPH; Brian P. Lucas, MD, MS; Oanh Nguyen, MD, MAS; Samir S. Shah, MD, MSCE; Erin E. Shaughnessy, MD, MSHCM; and Heidi J. Sucharew, PhD.

Research, in the field of Hospital Medicine, often leverages data collected for reasons other than research. For example, electronic medical record data or patient satisfaction survey results can be used to answer questions that are relevant to the practice of hospital medicine. In these types of datasets, data will inevitably be missing. Such missing data can compromise our ability to draw definitive conclusions from our research study. This review introduces the concept of missing data, describes patterns and mechanisms of missing data, and discusses common approaches for the handling of missing data, including sensitivity analyses for determining how robust the results are despite assumptions made about the missing data.

Missing data create a host of problems for researchers. First, missing data result in a loss of information and can diminish the power of the proposed study. Second, the irregular data complicate the analysis because many of the standard software procedures used have been developed for fully observed or “complete” data (ie, each subject has a value for all measures of interest). Finally, missing data may introduce bias due to the systematic difference between the observed and the unobserved data. For example, if men are less likely than women to complete all questions in a patient satisfaction survey when they are not satisfied, then hospital satisfaction analyses that rely on completed surveys would tend to provide biased estimates of the satisfaction males have with their care.

The ideal approach to mitigating problems caused by missing data is to anticipate and incorporate strategies to minimize missing data into the study design (ie, when planning data collection protocols for prospective studies). This plan should provide strategies for minimizing nonresponse and estimating the magnitude of anticipated missing data to ensure that the study achieves sufficient strength despite the missing data.

Strategies for minimizing nonresponse include (1) informing potential study participants, at initial contact, about the implications of missing data on the ability to answer the research question; (2) collecting several phone numbers, addresses, preferred method of contact and calling times, as well as an alternative contact, in case the primary study contact is unable to be reached; (3) specifying the number of call backs, as well as the time of contact; and (4) piloting data capture questions for phrasing, clarity, and sensitivity, in order to resolve problems before initiating the study. One approach that can be used to mitigate the impact of missing data in surveys is to contact a sample of the initial nonrespondents using a more intensive follow-up approach (eg, a nonresponse to a mailed survey is followed up by a telephone call in order to conduct the survey again over the phone), and this is referred to as “nonresponse two-phase sampling.” The additional data, captured in the second phase, not only reduces the nonresponse rate but can also provide important information on the missing data mechanism.^1,2 In longitudinal studies with dropouts, one can measure participants’ intent to drop out in order to evaluate how much the probability of dropping out depends on missing responses.³ One may also choose to determine the power and implications of sample size under different missing data assumptions.⁴

Different data sources are likely to have unique reasons for missing values due to the workflows involved in how the data are collected. In research involving the use of data from electronic medical records, missing data on specific diagnoses involving patients who are regularly engaged in care are often considered to be “not present” or “normal”, since clinical documentation workflows are largely governed by the concept of “documentation by exception” in which diagnoses are documented only when there is an exception to the expectation that these are not present. For example, “diabetes mellitus” is commonly documented, but “diabetes mellitus not present” is rarely documented in electronic medical records which are used for clinical care. Thus, lack of explicit documentation is likely to indicate that diabetes mellitus is, in fact, not present.

Certain variables may be missing simply because there is no quantifiable value­—ie, the data do not exist. Structural missingness refers to a value that does not exist for a logical reason (eg, “What is the gender of your first child?” for those who do not have a child). Censoring, which occurs during “time to event” analysis, refers to a situation where information about a subject stops before the event of interest happens, for example, when a subject in a study involving a 30-day outcome dies at day 14. The term “limit of detection” refers to the lowest or highest level at which two distinct values can reasonably be distinguished (eg, the lower limit of detection of a C-reactive protein assay may be 1 mg/dL, so lower values might simply be reported by the lab as <1 mg/dL).⁵ These types of missing data require specific methods that are not discussed in this review.

These examples illustrate that approaches to dealing with missing data vary depending on what data sources are used and how data are collected. Understanding the reasons missing data are present is a necessary step in formulating a robust analytic approach to handling missing data.

Missing Data Patterns

Evaluating missing data patterns provides information on the degree and complexity of the missing data problem and can aid in choosing an appropriate missing data handling method. This is because some analytic methods work well for a general pattern (nonmonotone) and other methods work for special patterns (eg, monotone, file matching). In longitudinal studies, missing data is commonly missing in a monotone pattern, where once one variable is missing then all subsequent variables are also missing for a particular subject. This occurs when a study participant is lost to follow-up. For example, a monotone missing data pattern may occur in a study that requires a series of follow-up visits for laboratory blood tests. If a patient drops out, it results in a monotone missing data pattern, as no data on blood test results are available once the patient drops out. If the patient just skips an intermediate visit but returns for the final blood test, this would show a nonmonotone missing data pattern. A file-matching pattern occurs when variables are never observed together. This pattern can occur when data from several studies are merged and some variables are not collected in all studies. For example, three studies are merged and all three collect blood pressure, but only one study collects age and only one study collects sex.

Missing Data Mechanisms

The missing data mechanism relates to the underlying reasons for missing values and the relationships between variables with and without missing data. In general, missing data can be either random or nonrandom with distinctions in randomness made by three types: (1) data missing completely at random (MCAR); (2) data missing at random (MAR); and (3) data missing not at random (MNAR).⁶ As with the missing data pattern, understanding the missing data mechanism can aid in selecting an appropriate approach to handling the missing data.

Data are MCAR if the missingness does not depend on any study variables, meaning that all subjects are equally likely to be missing certain data elements. When the data are MCAR, those with missing values can be viewed as a simple random sample from the complete (but never actually observed) data and can be dropped from analysis without causing bias in the results. If the values of some diagnostic tests were missing for some patients due to equipment malfunction or electricity outage, for example, then the missingness may be considered MCAR.

Data are MAR if the missingness depends on the observed characteristics but not the unobserved characteristics, meaning that the relationships observed in the data can be used to predict the occurrence of missing values. Because the “randomness” of MAR is conditional on observed characteristics, which distinguishes it from the “completely at random” type of MCAR, dropping or omitting those cases with missing values from the analysis may lead to biased results.⁷ In a study of quality of life (QOL) for patients with mild to moderate traumatic brain injury, if health-related QOL questions were not answered by some patients with high pain levels (even though the pain levels were recorded), the missingness of QOL may be considered as MAR. This is due to the fact that within subjects grouped by the observed characteristic of pain (that is, conditional on similar levels of pain) the missingness of QOL is the result of chance and does not depend on the values (observed or unobserved) of QOL. It follows then, that once grouped into a high (or low) pain stratum, if QOL is considered MAR, then, whether or not it is observed, is random.

Data are considered MNAR if their missingness depends on characteristics that are not observed and cannot be fully explained by the observed characteristics. Systematic differences between missing and nonmissing data exist for data that is MNAR. For example, if a survey of household income had an increased probability of missing incomes from the low-income families then the data would be considered as MNAR.

Randomness in the missing data mechanism may be ignored without affecting the inference in some circumstances.⁸ Both MCAR and MAR can be considered as “ignorable” in the sense that a proper method (eg, multiple imputation) may recover the missing information without modeling (ie, accounting for) the random process of the missing data mechanism (Table).⁹ In contrast, the MNAR mechanism requires a method that takes into account the missing data mechanism in order to make inferences about the complete (and partially unobserved) data; or in other words, a model for the missing data mechanism cannot be ignored. It is for this reason that the MNAR mechanism is often called “nonignorable”. Nonignorable missing data present a challenge to researchers because the mechanism underlying the missingness must be included in the analysis. Yet researchers rarely know what the missingness mechanism is, and the data needed to validate any putative mechanism is, in fact, missing. In cases when more than one variable is subject to missingness, researchers need to assess the missingness mechanism for each variable and tailor their approach to the specific missing data problems.⁹

There is no universally accepted standard to guide when statistical methods should be applied to account for missing data. The amount of missing data alone cannot fully assess the missing data problem; missing data patterns and mechanisms can have greater impact on research results than the proportion of missing data alone. A good statistical method for handling missing data should provide an unbiased estimate of the quantity that the investigators intend to estimate; make use of the partial information in the incomplete cases to improve efficiency (and in most cases also to reduce bias); and provide valid estimates of the standard errors, confidence intervals, and P values for statistical tests. There are generally four broadly defined classes of methods for handling missing data in clinical research: (1) the complete-case analysis, (2) single imputation methods, (3) the weighted estimating-equation approach, and (4) the model-based approach including maximum likelihood (ML) and multiple imputation (Table and Appendix).¹⁰

Since missing data mechanisms cannot be conclusively verified, it is good practice to conduct some sensitivity analyses to test the robustness of the primary results. For this purpose, pattern-mixture models provide a flexible framework for implementing sensitivity analyses to missing data assumptions and can be used to evaluate the possibility of the data being MNAR. In this framework, the missing data distribution is modeled and then incorporated into the outcome model of interest. Tipping-point analysis is a sensitivity analysis where the missing data is replaced with a range of values to determine how much the values must change for the results of the study to tip from significant to not significant. If the same general conclusions remain valid over a range of assumptions about the missing data values, then one can have greater confidence in the study conclusions.

In dealing with missing data from clinical research, clinicians and statisticians need to work together to minimize missingness at the data collection stage, document the reasons for missingness, use substantive knowledge, if possible, to assess the missing data mechanism, perform primary analysis based on a defensible missing data mechanism, and conduct a sensitivity analysis to assess whether the primary result is robust despite departure from the assumed missing data mechanism.

Acknowledgments

The following members of the Journal of Hospital Medicine Leadership team contributed to this review: Mel L. Anderson, MD; Peter Cram, MD, MBA; JoAnna K. Leyenaar, MD, PhD, MPH; Brian P. Lucas, MD, MS; Oanh Nguyen, MD, MAS; Samir S. Shah, MD, MSCE; Erin E. Shaughnessy, MD, MSHCM; and Heidi J. Sucharew, PhD.

Research, in the field of Hospital Medicine, often leverages data collected for reasons other than research. For example, electronic medical record data or patient satisfaction survey results can be used to answer questions that are relevant to the practice of hospital medicine. In these types of datasets, data will inevitably be missing. Such missing data can compromise our ability to draw definitive conclusions from our research study. This review introduces the concept of missing data, describes patterns and mechanisms of missing data, and discusses common approaches for the handling of missing data, including sensitivity analyses for determining how robust the results are despite assumptions made about the missing data.

Missing data create a host of problems for researchers. First, missing data result in a loss of information and can diminish the power of the proposed study. Second, the irregular data complicate the analysis because many of the standard software procedures used have been developed for fully observed or “complete” data (ie, each subject has a value for all measures of interest). Finally, missing data may introduce bias due to the systematic difference between the observed and the unobserved data. For example, if men are less likely than women to complete all questions in a patient satisfaction survey when they are not satisfied, then hospital satisfaction analyses that rely on completed surveys would tend to provide biased estimates of the satisfaction males have with their care.

The ideal approach to mitigating problems caused by missing data is to anticipate and incorporate strategies to minimize missing data into the study design (ie, when planning data collection protocols for prospective studies). This plan should provide strategies for minimizing nonresponse and estimating the magnitude of anticipated missing data to ensure that the study achieves sufficient strength despite the missing data.

Strategies for minimizing nonresponse include (1) informing potential study participants, at initial contact, about the implications of missing data on the ability to answer the research question; (2) collecting several phone numbers, addresses, preferred method of contact and calling times, as well as an alternative contact, in case the primary study contact is unable to be reached; (3) specifying the number of call backs, as well as the time of contact; and (4) piloting data capture questions for phrasing, clarity, and sensitivity, in order to resolve problems before initiating the study. One approach that can be used to mitigate the impact of missing data in surveys is to contact a sample of the initial nonrespondents using a more intensive follow-up approach (eg, a nonresponse to a mailed survey is followed up by a telephone call in order to conduct the survey again over the phone), and this is referred to as “nonresponse two-phase sampling.” The additional data, captured in the second phase, not only reduces the nonresponse rate but can also provide important information on the missing data mechanism.^1,2 In longitudinal studies with dropouts, one can measure participants’ intent to drop out in order to evaluate how much the probability of dropping out depends on missing responses.³ One may also choose to determine the power and implications of sample size under different missing data assumptions.⁴

Different data sources are likely to have unique reasons for missing values due to the workflows involved in how the data are collected. In research involving the use of data from electronic medical records, missing data on specific diagnoses involving patients who are regularly engaged in care are often considered to be “not present” or “normal”, since clinical documentation workflows are largely governed by the concept of “documentation by exception” in which diagnoses are documented only when there is an exception to the expectation that these are not present. For example, “diabetes mellitus” is commonly documented, but “diabetes mellitus not present” is rarely documented in electronic medical records which are used for clinical care. Thus, lack of explicit documentation is likely to indicate that diabetes mellitus is, in fact, not present.

Certain variables may be missing simply because there is no quantifiable value­—ie, the data do not exist. Structural missingness refers to a value that does not exist for a logical reason (eg, “What is the gender of your first child?” for those who do not have a child). Censoring, which occurs during “time to event” analysis, refers to a situation where information about a subject stops before the event of interest happens, for example, when a subject in a study involving a 30-day outcome dies at day 14. The term “limit of detection” refers to the lowest or highest level at which two distinct values can reasonably be distinguished (eg, the lower limit of detection of a C-reactive protein assay may be 1 mg/dL, so lower values might simply be reported by the lab as <1 mg/dL).⁵ These types of missing data require specific methods that are not discussed in this review.

These examples illustrate that approaches to dealing with missing data vary depending on what data sources are used and how data are collected. Understanding the reasons missing data are present is a necessary step in formulating a robust analytic approach to handling missing data.

Missing Data Patterns

Evaluating missing data patterns provides information on the degree and complexity of the missing data problem and can aid in choosing an appropriate missing data handling method. This is because some analytic methods work well for a general pattern (nonmonotone) and other methods work for special patterns (eg, monotone, file matching). In longitudinal studies, missing data is commonly missing in a monotone pattern, where once one variable is missing then all subsequent variables are also missing for a particular subject. This occurs when a study participant is lost to follow-up. For example, a monotone missing data pattern may occur in a study that requires a series of follow-up visits for laboratory blood tests. If a patient drops out, it results in a monotone missing data pattern, as no data on blood test results are available once the patient drops out. If the patient just skips an intermediate visit but returns for the final blood test, this would show a nonmonotone missing data pattern. A file-matching pattern occurs when variables are never observed together. This pattern can occur when data from several studies are merged and some variables are not collected in all studies. For example, three studies are merged and all three collect blood pressure, but only one study collects age and only one study collects sex.

Missing Data Mechanisms

The missing data mechanism relates to the underlying reasons for missing values and the relationships between variables with and without missing data. In general, missing data can be either random or nonrandom with distinctions in randomness made by three types: (1) data missing completely at random (MCAR); (2) data missing at random (MAR); and (3) data missing not at random (MNAR).⁶ As with the missing data pattern, understanding the missing data mechanism can aid in selecting an appropriate approach to handling the missing data.

Data are MCAR if the missingness does not depend on any study variables, meaning that all subjects are equally likely to be missing certain data elements. When the data are MCAR, those with missing values can be viewed as a simple random sample from the complete (but never actually observed) data and can be dropped from analysis without causing bias in the results. If the values of some diagnostic tests were missing for some patients due to equipment malfunction or electricity outage, for example, then the missingness may be considered MCAR.

Data are MAR if the missingness depends on the observed characteristics but not the unobserved characteristics, meaning that the relationships observed in the data can be used to predict the occurrence of missing values. Because the “randomness” of MAR is conditional on observed characteristics, which distinguishes it from the “completely at random” type of MCAR, dropping or omitting those cases with missing values from the analysis may lead to biased results.⁷ In a study of quality of life (QOL) for patients with mild to moderate traumatic brain injury, if health-related QOL questions were not answered by some patients with high pain levels (even though the pain levels were recorded), the missingness of QOL may be considered as MAR. This is due to the fact that within subjects grouped by the observed characteristic of pain (that is, conditional on similar levels of pain) the missingness of QOL is the result of chance and does not depend on the values (observed or unobserved) of QOL. It follows then, that once grouped into a high (or low) pain stratum, if QOL is considered MAR, then, whether or not it is observed, is random.

Data are considered MNAR if their missingness depends on characteristics that are not observed and cannot be fully explained by the observed characteristics. Systematic differences between missing and nonmissing data exist for data that is MNAR. For example, if a survey of household income had an increased probability of missing incomes from the low-income families then the data would be considered as MNAR.

Randomness in the missing data mechanism may be ignored without affecting the inference in some circumstances.⁸ Both MCAR and MAR can be considered as “ignorable” in the sense that a proper method (eg, multiple imputation) may recover the missing information without modeling (ie, accounting for) the random process of the missing data mechanism (Table).⁹ In contrast, the MNAR mechanism requires a method that takes into account the missing data mechanism in order to make inferences about the complete (and partially unobserved) data; or in other words, a model for the missing data mechanism cannot be ignored. It is for this reason that the MNAR mechanism is often called “nonignorable”. Nonignorable missing data present a challenge to researchers because the mechanism underlying the missingness must be included in the analysis. Yet researchers rarely know what the missingness mechanism is, and the data needed to validate any putative mechanism is, in fact, missing. In cases when more than one variable is subject to missingness, researchers need to assess the missingness mechanism for each variable and tailor their approach to the specific missing data problems.⁹

There is no universally accepted standard to guide when statistical methods should be applied to account for missing data. The amount of missing data alone cannot fully assess the missing data problem; missing data patterns and mechanisms can have greater impact on research results than the proportion of missing data alone. A good statistical method for handling missing data should provide an unbiased estimate of the quantity that the investigators intend to estimate; make use of the partial information in the incomplete cases to improve efficiency (and in most cases also to reduce bias); and provide valid estimates of the standard errors, confidence intervals, and P values for statistical tests. There are generally four broadly defined classes of methods for handling missing data in clinical research: (1) the complete-case analysis, (2) single imputation methods, (3) the weighted estimating-equation approach, and (4) the model-based approach including maximum likelihood (ML) and multiple imputation (Table and Appendix).¹⁰

Since missing data mechanisms cannot be conclusively verified, it is good practice to conduct some sensitivity analyses to test the robustness of the primary results. For this purpose, pattern-mixture models provide a flexible framework for implementing sensitivity analyses to missing data assumptions and can be used to evaluate the possibility of the data being MNAR. In this framework, the missing data distribution is modeled and then incorporated into the outcome model of interest. Tipping-point analysis is a sensitivity analysis where the missing data is replaced with a range of values to determine how much the values must change for the results of the study to tip from significant to not significant. If the same general conclusions remain valid over a range of assumptions about the missing data values, then one can have greater confidence in the study conclusions.

In dealing with missing data from clinical research, clinicians and statisticians need to work together to minimize missingness at the data collection stage, document the reasons for missingness, use substantive knowledge, if possible, to assess the missing data mechanism, perform primary analysis based on a defensible missing data mechanism, and conduct a sensitivity analysis to assess whether the primary result is robust despite departure from the assumed missing data mechanism.

Acknowledgments

The following members of the Journal of Hospital Medicine Leadership team contributed to this review: Mel L. Anderson, MD; Peter Cram, MD, MBA; JoAnna K. Leyenaar, MD, PhD, MPH; Brian P. Lucas, MD, MS; Oanh Nguyen, MD, MAS; Samir S. Shah, MD, MSCE; Erin E. Shaughnessy, MD, MSHCM; and Heidi J. Sucharew, PhD.

Hilda and I shared childhood stories while we enjoyed one of her favorite Mexican dishes, grilled nopalitos (cactus). Hilda loved nopalitos, but she rarely ate them because they are high in potassium. Hilda had end-stage kidney disease (ESKD), and as an undocumented Mexican immigrant in Denver, CO, she relied on emergency-only hemodialysis. Instead of receiving standard hemodialysis three times per week as required, Hilda would arrive critically ill to the hospital after her nausea, vomiting, and shortness of breath became unbearable. After three cardiac arrests from high potassium levels, she fervently avoided foods high in it. This time, however, she was not worried about potassium. This was our last meal together. She would fly to Mexico a few days later to die.

Our hospital medicine team knew Hilda well. We had continuity because we had been admitting her to the intensive care unit or medicine floor one night each week to receive two hemodialysis sessions when she was critically ill. I immediately connected with Hilda because our lives were parallel in many ways. Hilda and I were both in our early 30s, English was our second language, we both grew up in poverty, and we now had children in elementary school. I, however, was documented. My United States citizenship allowed me the privilege of pursuing a medical degree and gaining access to quality healthcare. In contrast, Hilda had been forced to end her education prematurely, marry her mother’s friend for financial stability at the age of 14, and eventually flee to the US to escape poverty. She survived by cleaning homes until her kidneys failed. Initially, Hilda was my patient. Over time, she became a dear friend.

The first two years of emergency-only hemodialysis devastated Hilda. Too sick to work, she became homeless, staying with a nurse until we found a shelter for single mothers. Multiple cardiac arrests and resuscitations traumatized her young sons, who called 911 each time she collapsed and witnessed the resuscitations. Her boys did not understand the cycle of separation from their mother for her emergent, weekly dialysis hospital admissions and wondered if she would survive to the following week. After two years of emergency-only dialysis, Hilda’s deep love for her boys and concern about the possibility that her sudden death could leave them alone led her to pre-emptively decide to stop emergency-only dialysis. Had Hilda’s treatment costs been covered by emergency Medicaid, as undocumented immigrants with ESKD are in some other states, she may not have been forced into this terrible decision. Moving to a state where standard dialysis is covered was not an option for Hilda because she wanted her boys to stay in Colorado where they had family and friends. With no other options, she first sought a loving adoptive family in the US so that her boys could grow up and have the opportunity to pursue an education. After carefully finding the right adoptive parents, Hilda wanted to celebrate her life with the people she loved. To show her gratitude, she organized a large Mexican Christmas party and invited all of the healthcare providers and friends that had supported her. She generously gave everyone a small gift to remember her by from the few things she owned. I received the wooden rosary her father had left her. A short while later, Hilda flew home to Mexico and passed away on Mother’s Day in 2014.

Two years of caring for Hilda as an internal medicine hospitalist changed me. Grief gave way to anger, anger to determination. I found it morally distressing to continue to provide this type of care. Something had to change and there was little research in this area. One small study had demonstrated that emergency-only hemodialysis was nearly four-fold more expensive due to additional visits to the emergency department and admissions to the hospital, compared to standard outpatient hemodialysis.¹ After much soul-searching and advice seeking, I scaled down my clinical hospitalist shifts and gathered a team to do research. For four years, we worked on illuminating the suffering of undocumented immigrants with ESKD that rely on emergency-only hemodialysis. We conducted 20 individual face-to-face qualitative interviews with undocumented immigrants with ESKD and heard first-hand about the emotional and physical burdens and the existential anxiety associated with weekly threats to life.² We published a retrospective cohort study looking at differences in mortality and found that immigrants who relied on emergency-only hemodialysis had a 14-fold greater mortality rate than those on standard hemodialysis five years after initiating hemodialysis.³ In another retrospective study, we described the circumstances among undocumented immigrants with ESKD who died in the hospital after presenting with ESKD complications, and found that the majority presented with high potassium and a recorded rhythm disturbance.⁴ I discovered that as a hospitalist physician, I was not the only one distressed. We conducted 50 qualitative interviews to determine the perspectives of interdisciplinary clinicians on providing emergency dialysis and found that there are more clinicians experiencing moral distress. They described several important drivers of burnout,⁵ including emotional exhaustion from witnessing needless suffering and high mortality, as well as physical exhaustion from overextending themselves to bridge their patient’s care. Together, we discovered that the research told the larger narrative behind Hilda’s struggles. These publications caught the attention of the media and enabled us to speak to a wider audience of clinicians, health policy makers, and the general public.^6-10 They also became a catalyst to engaging and enlisting the good will and interest of a number of key stakeholders to look for solutions.

In the US, undocumented immigrants do not qualify for insurance through traditional Medicaid, Medicare, or the provisions from the Patient Protection and Affordable Care Act. Instead, emergency Medicaid provides reimbursements for care of undocumented immigrants. According to the 1986 Emergency Medicaid Treatment and Active Labor Act, federal Medicaid payments can only be made for the care of undocumented immigrants if care is necessary for the treatment of an emergency medical condition.¹¹ However, the Centers for Medicare and Medicaid (CMS) has outlined certain conditions that cannot qualify for matching federal funds under emergency Medicaid (ie, organ transplant and routine prenatal or postpartum care). Beyond these requirements, federal CMS and the Office of the Inspector General defer to states to define what constitutes a medical emergency. A few states include ESKD in the definition of “emergency medical condition,” thereby expanding access to standard hemodialysis to undocumented immigrants. We wanted Colorado to join that list.

On August 2018, after four years of research and months of dialog, everything changed: Colorado Medicaid announced that ESKD was now an “emergency medical condition.” As simple as that, undocumented immigrants would receive standard maintenance hemodialysis. Tears streamed down my face as I read a message from a policy specialist from the Colorado Medicaid: Your team “played a big role in bringing awareness to this issue, and your advocacy for these patients is impressive … thank you for fighting for such an important cause.” I reread her message, imagining what this would have meant to Hilda and her boys.

Our work to enhance care in this community is not over. To better understand the provision of dialysis care for undocumented immigrants in the United States, our team reviewed the Medicaid language for each of the 50 US states in addition to connecting with clinicians and organizations (eg, National Kidney Foundation and ESKD Networks). We found that only 12 states provide Medicaid reimbursement for standard dialysis and that a majority of the US states do not currently define need for dialysis as an emergency medical condition.¹² As our Colorado team works with stakeholders in other states interested in similarly redefining their state’s emergency Medicaid definition, our most important advice is that advocacy is a team-based effort. There may be resistance and some may argue that expanding access to care would be an economic burden on taxpayers; however, research demonstrates that undocumented immigrants contribute more to the US Medicare Trust Fund than they actually withdraw toward healthcare.¹³ Furthermore, a new study has demonstrated that a net savings of nearly $6,000 per person per month is realized when patients are transitioned from emergency-only hemodialysis to standard hemodialysis.¹⁴

Internal medicine hospitalists on the front-line of healthcare systems are regular witnesses to its horrible injustices. We rarely share our perspectives and do not expect change to follow. With Hilda, we saw how a powerful combination of research and coalition building could lift one patient’s tragic story to a level where it could produce change. Augmenting Hilda’s experience of tragically poor access to care with evidence-based research gave her story validity far beyond our immediate circle of friends and colleagues, making a singular tragedy, policy relevant. Each time we shared our research to community advocacy groups, health policy stakeholders, state legislators, nurses, and staff; we began with Hilda’s story, not just because it inspired us, but because its truth was undeniable. Our patients’ stories matter, and it is our responsibility to tell them.

Each time I prepare nopalitos for my family, I think of my last meal with Hilda. No matter how painful or difficult her struggle with ESKD, Hilda persisted. She protected her boys. They were her purpose. When she knew she could no longer give them the life she wanted for them, she found a family who would. Hilda’s sons now live with a loving adoptive family, are thriving in school, and her oldest is interested in becoming a physician. Nopal, or cactus, symbolizes such endurance—a plant with unique adaptations and strength that can flourish under extreme environmental stress. Like a cactus storing precious water, Hilda treasured her children, and her resolve to provide for them was unstoppable, right to the edge of death. When our team first took up Hilda’s cause, change seemed impossible. We discovered the opposite. As I clench the wooden rosary she left me that Christmas, I thank her for giving our team the courage to adapt and persist, for in doing so we found a path, first to research and then to broader partnerships and more meaningful policy changes.

Acknowledgments

The author would like to thank Hilda, her family, and the patients at Denver Health. She would also like to acknowledge Hilda’s family, Drs. Mark Earnest, John F. Steiner, Romana Hasnain-Wynia, Rudolph Rodriguez, Judy Regensteiner, and Michel Chonchol for reading and providing feedback on earlier drafts of this narrative.

Hilda and I shared childhood stories while we enjoyed one of her favorite Mexican dishes, grilled nopalitos (cactus). Hilda loved nopalitos, but she rarely ate them because they are high in potassium. Hilda had end-stage kidney disease (ESKD), and as an undocumented Mexican immigrant in Denver, CO, she relied on emergency-only hemodialysis. Instead of receiving standard hemodialysis three times per week as required, Hilda would arrive critically ill to the hospital after her nausea, vomiting, and shortness of breath became unbearable. After three cardiac arrests from high potassium levels, she fervently avoided foods high in it. This time, however, she was not worried about potassium. This was our last meal together. She would fly to Mexico a few days later to die.

Our hospital medicine team knew Hilda well. We had continuity because we had been admitting her to the intensive care unit or medicine floor one night each week to receive two hemodialysis sessions when she was critically ill. I immediately connected with Hilda because our lives were parallel in many ways. Hilda and I were both in our early 30s, English was our second language, we both grew up in poverty, and we now had children in elementary school. I, however, was documented. My United States citizenship allowed me the privilege of pursuing a medical degree and gaining access to quality healthcare. In contrast, Hilda had been forced to end her education prematurely, marry her mother’s friend for financial stability at the age of 14, and eventually flee to the US to escape poverty. She survived by cleaning homes until her kidneys failed. Initially, Hilda was my patient. Over time, she became a dear friend.

The first two years of emergency-only hemodialysis devastated Hilda. Too sick to work, she became homeless, staying with a nurse until we found a shelter for single mothers. Multiple cardiac arrests and resuscitations traumatized her young sons, who called 911 each time she collapsed and witnessed the resuscitations. Her boys did not understand the cycle of separation from their mother for her emergent, weekly dialysis hospital admissions and wondered if she would survive to the following week. After two years of emergency-only dialysis, Hilda’s deep love for her boys and concern about the possibility that her sudden death could leave them alone led her to pre-emptively decide to stop emergency-only dialysis. Had Hilda’s treatment costs been covered by emergency Medicaid, as undocumented immigrants with ESKD are in some other states, she may not have been forced into this terrible decision. Moving to a state where standard dialysis is covered was not an option for Hilda because she wanted her boys to stay in Colorado where they had family and friends. With no other options, she first sought a loving adoptive family in the US so that her boys could grow up and have the opportunity to pursue an education. After carefully finding the right adoptive parents, Hilda wanted to celebrate her life with the people she loved. To show her gratitude, she organized a large Mexican Christmas party and invited all of the healthcare providers and friends that had supported her. She generously gave everyone a small gift to remember her by from the few things she owned. I received the wooden rosary her father had left her. A short while later, Hilda flew home to Mexico and passed away on Mother’s Day in 2014.

Two years of caring for Hilda as an internal medicine hospitalist changed me. Grief gave way to anger, anger to determination. I found it morally distressing to continue to provide this type of care. Something had to change and there was little research in this area. One small study had demonstrated that emergency-only hemodialysis was nearly four-fold more expensive due to additional visits to the emergency department and admissions to the hospital, compared to standard outpatient hemodialysis.¹ After much soul-searching and advice seeking, I scaled down my clinical hospitalist shifts and gathered a team to do research. For four years, we worked on illuminating the suffering of undocumented immigrants with ESKD that rely on emergency-only hemodialysis. We conducted 20 individual face-to-face qualitative interviews with undocumented immigrants with ESKD and heard first-hand about the emotional and physical burdens and the existential anxiety associated with weekly threats to life.² We published a retrospective cohort study looking at differences in mortality and found that immigrants who relied on emergency-only hemodialysis had a 14-fold greater mortality rate than those on standard hemodialysis five years after initiating hemodialysis.³ In another retrospective study, we described the circumstances among undocumented immigrants with ESKD who died in the hospital after presenting with ESKD complications, and found that the majority presented with high potassium and a recorded rhythm disturbance.⁴ I discovered that as a hospitalist physician, I was not the only one distressed. We conducted 50 qualitative interviews to determine the perspectives of interdisciplinary clinicians on providing emergency dialysis and found that there are more clinicians experiencing moral distress. They described several important drivers of burnout,⁵ including emotional exhaustion from witnessing needless suffering and high mortality, as well as physical exhaustion from overextending themselves to bridge their patient’s care. Together, we discovered that the research told the larger narrative behind Hilda’s struggles. These publications caught the attention of the media and enabled us to speak to a wider audience of clinicians, health policy makers, and the general public.^6-10 They also became a catalyst to engaging and enlisting the good will and interest of a number of key stakeholders to look for solutions.

In the US, undocumented immigrants do not qualify for insurance through traditional Medicaid, Medicare, or the provisions from the Patient Protection and Affordable Care Act. Instead, emergency Medicaid provides reimbursements for care of undocumented immigrants. According to the 1986 Emergency Medicaid Treatment and Active Labor Act, federal Medicaid payments can only be made for the care of undocumented immigrants if care is necessary for the treatment of an emergency medical condition.¹¹ However, the Centers for Medicare and Medicaid (CMS) has outlined certain conditions that cannot qualify for matching federal funds under emergency Medicaid (ie, organ transplant and routine prenatal or postpartum care). Beyond these requirements, federal CMS and the Office of the Inspector General defer to states to define what constitutes a medical emergency. A few states include ESKD in the definition of “emergency medical condition,” thereby expanding access to standard hemodialysis to undocumented immigrants. We wanted Colorado to join that list.

On August 2018, after four years of research and months of dialog, everything changed: Colorado Medicaid announced that ESKD was now an “emergency medical condition.” As simple as that, undocumented immigrants would receive standard maintenance hemodialysis. Tears streamed down my face as I read a message from a policy specialist from the Colorado Medicaid: Your team “played a big role in bringing awareness to this issue, and your advocacy for these patients is impressive … thank you for fighting for such an important cause.” I reread her message, imagining what this would have meant to Hilda and her boys.

Our work to enhance care in this community is not over. To better understand the provision of dialysis care for undocumented immigrants in the United States, our team reviewed the Medicaid language for each of the 50 US states in addition to connecting with clinicians and organizations (eg, National Kidney Foundation and ESKD Networks). We found that only 12 states provide Medicaid reimbursement for standard dialysis and that a majority of the US states do not currently define need for dialysis as an emergency medical condition.¹² As our Colorado team works with stakeholders in other states interested in similarly redefining their state’s emergency Medicaid definition, our most important advice is that advocacy is a team-based effort. There may be resistance and some may argue that expanding access to care would be an economic burden on taxpayers; however, research demonstrates that undocumented immigrants contribute more to the US Medicare Trust Fund than they actually withdraw toward healthcare.¹³ Furthermore, a new study has demonstrated that a net savings of nearly $6,000 per person per month is realized when patients are transitioned from emergency-only hemodialysis to standard hemodialysis.¹⁴

Internal medicine hospitalists on the front-line of healthcare systems are regular witnesses to its horrible injustices. We rarely share our perspectives and do not expect change to follow. With Hilda, we saw how a powerful combination of research and coalition building could lift one patient’s tragic story to a level where it could produce change. Augmenting Hilda’s experience of tragically poor access to care with evidence-based research gave her story validity far beyond our immediate circle of friends and colleagues, making a singular tragedy, policy relevant. Each time we shared our research to community advocacy groups, health policy stakeholders, state legislators, nurses, and staff; we began with Hilda’s story, not just because it inspired us, but because its truth was undeniable. Our patients’ stories matter, and it is our responsibility to tell them.

Each time I prepare nopalitos for my family, I think of my last meal with Hilda. No matter how painful or difficult her struggle with ESKD, Hilda persisted. She protected her boys. They were her purpose. When she knew she could no longer give them the life she wanted for them, she found a family who would. Hilda’s sons now live with a loving adoptive family, are thriving in school, and her oldest is interested in becoming a physician. Nopal, or cactus, symbolizes such endurance—a plant with unique adaptations and strength that can flourish under extreme environmental stress. Like a cactus storing precious water, Hilda treasured her children, and her resolve to provide for them was unstoppable, right to the edge of death. When our team first took up Hilda’s cause, change seemed impossible. We discovered the opposite. As I clench the wooden rosary she left me that Christmas, I thank her for giving our team the courage to adapt and persist, for in doing so we found a path, first to research and then to broader partnerships and more meaningful policy changes.

Acknowledgments

The author would like to thank Hilda, her family, and the patients at Denver Health. She would also like to acknowledge Hilda’s family, Drs. Mark Earnest, John F. Steiner, Romana Hasnain-Wynia, Rudolph Rodriguez, Judy Regensteiner, and Michel Chonchol for reading and providing feedback on earlier drafts of this narrative.

Hilda and I shared childhood stories while we enjoyed one of her favorite Mexican dishes, grilled nopalitos (cactus). Hilda loved nopalitos, but she rarely ate them because they are high in potassium. Hilda had end-stage kidney disease (ESKD), and as an undocumented Mexican immigrant in Denver, CO, she relied on emergency-only hemodialysis. Instead of receiving standard hemodialysis three times per week as required, Hilda would arrive critically ill to the hospital after her nausea, vomiting, and shortness of breath became unbearable. After three cardiac arrests from high potassium levels, she fervently avoided foods high in it. This time, however, she was not worried about potassium. This was our last meal together. She would fly to Mexico a few days later to die.

Our hospital medicine team knew Hilda well. We had continuity because we had been admitting her to the intensive care unit or medicine floor one night each week to receive two hemodialysis sessions when she was critically ill. I immediately connected with Hilda because our lives were parallel in many ways. Hilda and I were both in our early 30s, English was our second language, we both grew up in poverty, and we now had children in elementary school. I, however, was documented. My United States citizenship allowed me the privilege of pursuing a medical degree and gaining access to quality healthcare. In contrast, Hilda had been forced to end her education prematurely, marry her mother’s friend for financial stability at the age of 14, and eventually flee to the US to escape poverty. She survived by cleaning homes until her kidneys failed. Initially, Hilda was my patient. Over time, she became a dear friend.

The first two years of emergency-only hemodialysis devastated Hilda. Too sick to work, she became homeless, staying with a nurse until we found a shelter for single mothers. Multiple cardiac arrests and resuscitations traumatized her young sons, who called 911 each time she collapsed and witnessed the resuscitations. Her boys did not understand the cycle of separation from their mother for her emergent, weekly dialysis hospital admissions and wondered if she would survive to the following week. After two years of emergency-only dialysis, Hilda’s deep love for her boys and concern about the possibility that her sudden death could leave them alone led her to pre-emptively decide to stop emergency-only dialysis. Had Hilda’s treatment costs been covered by emergency Medicaid, as undocumented immigrants with ESKD are in some other states, she may not have been forced into this terrible decision. Moving to a state where standard dialysis is covered was not an option for Hilda because she wanted her boys to stay in Colorado where they had family and friends. With no other options, she first sought a loving adoptive family in the US so that her boys could grow up and have the opportunity to pursue an education. After carefully finding the right adoptive parents, Hilda wanted to celebrate her life with the people she loved. To show her gratitude, she organized a large Mexican Christmas party and invited all of the healthcare providers and friends that had supported her. She generously gave everyone a small gift to remember her by from the few things she owned. I received the wooden rosary her father had left her. A short while later, Hilda flew home to Mexico and passed away on Mother’s Day in 2014.

Two years of caring for Hilda as an internal medicine hospitalist changed me. Grief gave way to anger, anger to determination. I found it morally distressing to continue to provide this type of care. Something had to change and there was little research in this area. One small study had demonstrated that emergency-only hemodialysis was nearly four-fold more expensive due to additional visits to the emergency department and admissions to the hospital, compared to standard outpatient hemodialysis.¹ After much soul-searching and advice seeking, I scaled down my clinical hospitalist shifts and gathered a team to do research. For four years, we worked on illuminating the suffering of undocumented immigrants with ESKD that rely on emergency-only hemodialysis. We conducted 20 individual face-to-face qualitative interviews with undocumented immigrants with ESKD and heard first-hand about the emotional and physical burdens and the existential anxiety associated with weekly threats to life.² We published a retrospective cohort study looking at differences in mortality and found that immigrants who relied on emergency-only hemodialysis had a 14-fold greater mortality rate than those on standard hemodialysis five years after initiating hemodialysis.³ In another retrospective study, we described the circumstances among undocumented immigrants with ESKD who died in the hospital after presenting with ESKD complications, and found that the majority presented with high potassium and a recorded rhythm disturbance.⁴ I discovered that as a hospitalist physician, I was not the only one distressed. We conducted 50 qualitative interviews to determine the perspectives of interdisciplinary clinicians on providing emergency dialysis and found that there are more clinicians experiencing moral distress. They described several important drivers of burnout,⁵ including emotional exhaustion from witnessing needless suffering and high mortality, as well as physical exhaustion from overextending themselves to bridge their patient’s care. Together, we discovered that the research told the larger narrative behind Hilda’s struggles. These publications caught the attention of the media and enabled us to speak to a wider audience of clinicians, health policy makers, and the general public.^6-10 They also became a catalyst to engaging and enlisting the good will and interest of a number of key stakeholders to look for solutions.

In the US, undocumented immigrants do not qualify for insurance through traditional Medicaid, Medicare, or the provisions from the Patient Protection and Affordable Care Act. Instead, emergency Medicaid provides reimbursements for care of undocumented immigrants. According to the 1986 Emergency Medicaid Treatment and Active Labor Act, federal Medicaid payments can only be made for the care of undocumented immigrants if care is necessary for the treatment of an emergency medical condition.¹¹ However, the Centers for Medicare and Medicaid (CMS) has outlined certain conditions that cannot qualify for matching federal funds under emergency Medicaid (ie, organ transplant and routine prenatal or postpartum care). Beyond these requirements, federal CMS and the Office of the Inspector General defer to states to define what constitutes a medical emergency. A few states include ESKD in the definition of “emergency medical condition,” thereby expanding access to standard hemodialysis to undocumented immigrants. We wanted Colorado to join that list.

On August 2018, after four years of research and months of dialog, everything changed: Colorado Medicaid announced that ESKD was now an “emergency medical condition.” As simple as that, undocumented immigrants would receive standard maintenance hemodialysis. Tears streamed down my face as I read a message from a policy specialist from the Colorado Medicaid: Your team “played a big role in bringing awareness to this issue, and your advocacy for these patients is impressive … thank you for fighting for such an important cause.” I reread her message, imagining what this would have meant to Hilda and her boys.

Our work to enhance care in this community is not over. To better understand the provision of dialysis care for undocumented immigrants in the United States, our team reviewed the Medicaid language for each of the 50 US states in addition to connecting with clinicians and organizations (eg, National Kidney Foundation and ESKD Networks). We found that only 12 states provide Medicaid reimbursement for standard dialysis and that a majority of the US states do not currently define need for dialysis as an emergency medical condition.¹² As our Colorado team works with stakeholders in other states interested in similarly redefining their state’s emergency Medicaid definition, our most important advice is that advocacy is a team-based effort. There may be resistance and some may argue that expanding access to care would be an economic burden on taxpayers; however, research demonstrates that undocumented immigrants contribute more to the US Medicare Trust Fund than they actually withdraw toward healthcare.¹³ Furthermore, a new study has demonstrated that a net savings of nearly $6,000 per person per month is realized when patients are transitioned from emergency-only hemodialysis to standard hemodialysis.¹⁴

Internal medicine hospitalists on the front-line of healthcare systems are regular witnesses to its horrible injustices. We rarely share our perspectives and do not expect change to follow. With Hilda, we saw how a powerful combination of research and coalition building could lift one patient’s tragic story to a level where it could produce change. Augmenting Hilda’s experience of tragically poor access to care with evidence-based research gave her story validity far beyond our immediate circle of friends and colleagues, making a singular tragedy, policy relevant. Each time we shared our research to community advocacy groups, health policy stakeholders, state legislators, nurses, and staff; we began with Hilda’s story, not just because it inspired us, but because its truth was undeniable. Our patients’ stories matter, and it is our responsibility to tell them.

Each time I prepare nopalitos for my family, I think of my last meal with Hilda. No matter how painful or difficult her struggle with ESKD, Hilda persisted. She protected her boys. They were her purpose. When she knew she could no longer give them the life she wanted for them, she found a family who would. Hilda’s sons now live with a loving adoptive family, are thriving in school, and her oldest is interested in becoming a physician. Nopal, or cactus, symbolizes such endurance—a plant with unique adaptations and strength that can flourish under extreme environmental stress. Like a cactus storing precious water, Hilda treasured her children, and her resolve to provide for them was unstoppable, right to the edge of death. When our team first took up Hilda’s cause, change seemed impossible. We discovered the opposite. As I clench the wooden rosary she left me that Christmas, I thank her for giving our team the courage to adapt and persist, for in doing so we found a path, first to research and then to broader partnerships and more meaningful policy changes.

Acknowledgments

The author would like to thank Hilda, her family, and the patients at Denver Health. She would also like to acknowledge Hilda’s family, Drs. Mark Earnest, John F. Steiner, Romana Hasnain-Wynia, Rudolph Rodriguez, Judy Regensteiner, and Michel Chonchol for reading and providing feedback on earlier drafts of this narrative.

Internists are experts in general medicine, skilled at mapping the few hundred ways the human body can go awry onto thousands of diagnoses, and managing the uncertainty inherent in that process. Generalists, almost by definition, consult specialists with their specialty-focused questions; but who does one call for a general consultation about diagnosis if a specific diagnosis remains elusive and the pathology does not fit cleanly into the purview of a consultant? Outside of sage advice from colleagues (usually senior), most medical centers lack a consultation service focused on diagnosis. There is no oracle to seek. In this perspective, we describe our institution’s answer to this problem: the creation of a service for difficult diagnosis based on Socratic principles, particularly the role of iterative hypothesis testing in the process of diagnosis.¹

In 2015, Northwestern Medicine began the Socrates Project, a physician-to-physician consultation service that assists doctors working to diagnose conditions that have so far eluded detection. Our service’s goal is to improve patient care by providing an opinion to the referring physician on diagnostic possibilities for a particular case and ideas to reduce—or at least manage—diagnostic uncertainty.

Most patients referred to the Socrates Project have already undergone an extensive evaluation at top medical centers by experienced clinicians. It would be hubris to assume that we will find a definitive diagnosis in every case; indeed, because of the types of cases referred to our group, it is rare that we find a “Eureka!” diagnosis. When a colleague consults our group, we under-promise in hopes of over-delivering. Instead, we convey to referring physicians that we will conduct a thorough case review and explain our thinking in hopes of uncovering an additional diagnostic avenue, even if that avenue does not ultimately lead to a definitive diagnosis. In addition, the Socrates Project often serves as a broker between consulting services that are deadlocked because of differing diagnostic opinions. We also assist with cases in which a functional disorder is suspected, yet the referring physician is hesitant to diagnose a patient with such a disorder out of concern about missing an important (and possibly obscure) diagnosis.

The Socrates Project receives approximately two consult requests per week, usually from general internists but also from specialists in nearly all disciplines. Around 80% of the referrals are for current inpatients. Our service model is similar to a tumor board, which exists as an interdisciplinary group operating in parallel to the clinical services, to provide consensus-based recommendations. As a result, we act as doctors for doctors, formalizing the curbside consultation. Our usual turnaround time is a week but can be faster for urgent cases. Currently, Socrates Project members, including the faculty leader, volunteer their time and effort at no cost, and there are no charges to patients when physicians consult our group. An overview of the Socrates Project’s personnel and process are outlined in the Figure.

Northwestern’s Chief Medical Residents (CMRs) serve as the fellows for the service, and one of them assumes primary responsibility for each new consultation request the service receives. After obtaining the patient’s case history from the referring provider, the CMR then undertakes a thorough review of the electronic health record and any other available records from other institutions. In the inpatient setting, the CMR performs a new history and physical; phone calls or video conferencing permit history taking for outpatients. In contrast with the standard consultant note, we do not redocument the history, physical, and lab and imaging findings but instead construct a detailed problem list that synthesizes relevant findings into a useful working document.

The service’s faculty leader (BDS) then reviews the problem list with the CMRs to help refine the problem list and begin producing a differential diagnosis during a weekly hour-long meeting. As evidence supports team-based diagnostic collaborations,² the problem list and preliminary differential diagnosis then becomes a shareable document that the CMR or team leader presents to ad hoc general internists, specialists, and the other CMRs. The presentation can be in person, by phone, or e-mail. These ad hoc members, approximately 20 in number and spanning from junior attending physicians to senior clinicians, have volunteered to help the Socrates Project by adding their thoughts on differential diagnoses that explain the problem list and how to move forward with further testing. The ad hoc members have self-identified as clinicians with an interest in medical diagnosis—including surgeons, neurologists, psychiatrists, radiologists, and pathologists—and range in expertise from general internists to subspecialists. Finally, we document our problem list, differential diagnosis, and recommendations in the medical record and discuss the case with the referring team. The service limits its scope of clinical recommendation to diagnosis and avoids commenting on management decisions outside of the use of therapies as empiric diagnostic tests. A sample note is provided as an online Appendix.

Despite our process, we are often left without a satisfying diagnosis. We then are then faced with three possibilities: (1) The diagnosis is identifiable, just not by the physicians involved in the case—we did not think of the diagnosis in our deliberations; (2) The diagnosis is a described condition but without an available test—autoimmune limbic encephalitis associated with an unassayable or unknown auto-antibody, or the acuity of a critically ill patient makes diagnostic testing unreliable or not feasible; (3) The diagnosis has not yet been described by medical science—we are seeing a case of HIV infection in 1971.

With the personnel and process outlined above, we hope to provide recommendations that are useful in guiding a diagnostic workup regardless of which of these three scenarios is applicable. Our flexibility with involving the appropriate specialists in the Socrates Project should minimize the number of patients with a knowable diagnosis that is unknown to us. In the second scenario, our recommendations may rest upon the incorporation of a treatment as a diagnostic test. In the limbic encephalitis example above, a trial of steroids with rapid improvement in the patient’s condition may increase diagnostic certainty. The third scenario is the most difficult to identify. Pattern recognition of similarly presenting patients, keeping ourselves updated on pertinent primary literature, and consideration of advanced diagnostic testing such as exome sequencing and other next-generation sequencing strategies are essential in hoping to characterize a specific clinical syndrome that has yet to be described.

For situations in which our recommendations do not yield a diagnosis, we recognize the role for protocols such as genomic or metagenomic sequencing that assess multiple diagnostic possibilities in parallel without an a priori hypothesis.^3,4 The utility of multi-omics testing in diagnostic workups has been detailed by the Undiagnosed Diseases Network (UDN), which has created a systematic approach to describing new syndromes with the aid of metabolomic and genomic profiling.⁵ It is important to note that even with the resources available to the UDN, the diagnosis rate is 35%, emphasizing that in the majority of diagnosis-refractory cases, a diagnosis will not be found. This low diagnosis rate underscores the need for continued inquiry and cataloging of cases and data for further review or synthesis as the body of medical knowledge continues to expand. For these reasons, we have a follow-up system in place, which involves the assigned CMR regularly reviewing the chart and reporting during our weekly meetings. We make phone calls to patients and providers for cases that appear to be lost to follow-up.

We recognize several important limitations to our care model that may represent barriers to establishing, maintaining, and evaluating a similar service at other institutions. For example, there are limitations and benefits of the CMR as point person for managing our consultations. While they are admittedly junior colleagues with limited experience, CMRs tend to be among the best-read and up-to-date clinicians in the hospital by virtue of their recent general-medicine training and identification as a top clinician and leader. Moreover, in their role with the Socrates Project, CMRs have more time to think, talk with patients, and review the medical record than other clinicians, who may be under pressure to see an increasing number of patients while billing at higher levels. Indeed, the Socrates Project CMRs have, on a number of occasions, been the team members who find the piece of data that no one else thought relevant.

Another factor that may limit establishment of a similar team at other institutions is our volunteer-based model. The Socrates Project members volunteer because they love clinical medicine and serve on the team without remuneration for professional effort. With the CMR role as a notable exception, pressure from achieving relative value unit targets, obtaining grant funding, and publishing primary research publications in their field may limit this care model, particularly when shifting from a clinical-only activity to one that also formally investigates the service’s process and outcomes.

Beyond our clinical objective, we hope that the Socrates Project will further the discovery and description of previously unrecognized disease processes. To that end, we are pursuing an institutional review board-approved protocol to perform a rigorous assessment of the Socrates Project’s process and outcomes, including a cataloging of case archetypes and the time to definitive diagnosis if a diagnosis is established. As we continue to collect data, increasing our referral network may also lead to refinement and improvement in diagnostic processes and outcomes. Over time, we expect that the diagnostic resources available to us will evolve. Utilizing collective intelligence has been shown to improve diagnostic accuracy,⁶ and emerging artificial intelligence technologies may improve diagnostic performance as well.^7,8 Most importantly, through this endeavor, we hope to serve less as an oracle and more as a humble Socratic consultant for clinicians working to reduce diagnostic uncertainty for their patients.

Acknowledgments

The authors wish to thank the Northwestern University Chief Medical Residents, 2015-present, for their tireless efforts in support of the Socrates Project.

Internists are experts in general medicine, skilled at mapping the few hundred ways the human body can go awry onto thousands of diagnoses, and managing the uncertainty inherent in that process. Generalists, almost by definition, consult specialists with their specialty-focused questions; but who does one call for a general consultation about diagnosis if a specific diagnosis remains elusive and the pathology does not fit cleanly into the purview of a consultant? Outside of sage advice from colleagues (usually senior), most medical centers lack a consultation service focused on diagnosis. There is no oracle to seek. In this perspective, we describe our institution’s answer to this problem: the creation of a service for difficult diagnosis based on Socratic principles, particularly the role of iterative hypothesis testing in the process of diagnosis.¹

In 2015, Northwestern Medicine began the Socrates Project, a physician-to-physician consultation service that assists doctors working to diagnose conditions that have so far eluded detection. Our service’s goal is to improve patient care by providing an opinion to the referring physician on diagnostic possibilities for a particular case and ideas to reduce—or at least manage—diagnostic uncertainty.

Most patients referred to the Socrates Project have already undergone an extensive evaluation at top medical centers by experienced clinicians. It would be hubris to assume that we will find a definitive diagnosis in every case; indeed, because of the types of cases referred to our group, it is rare that we find a “Eureka!” diagnosis. When a colleague consults our group, we under-promise in hopes of over-delivering. Instead, we convey to referring physicians that we will conduct a thorough case review and explain our thinking in hopes of uncovering an additional diagnostic avenue, even if that avenue does not ultimately lead to a definitive diagnosis. In addition, the Socrates Project often serves as a broker between consulting services that are deadlocked because of differing diagnostic opinions. We also assist with cases in which a functional disorder is suspected, yet the referring physician is hesitant to diagnose a patient with such a disorder out of concern about missing an important (and possibly obscure) diagnosis.

The Socrates Project receives approximately two consult requests per week, usually from general internists but also from specialists in nearly all disciplines. Around 80% of the referrals are for current inpatients. Our service model is similar to a tumor board, which exists as an interdisciplinary group operating in parallel to the clinical services, to provide consensus-based recommendations. As a result, we act as doctors for doctors, formalizing the curbside consultation. Our usual turnaround time is a week but can be faster for urgent cases. Currently, Socrates Project members, including the faculty leader, volunteer their time and effort at no cost, and there are no charges to patients when physicians consult our group. An overview of the Socrates Project’s personnel and process are outlined in the Figure.

Northwestern’s Chief Medical Residents (CMRs) serve as the fellows for the service, and one of them assumes primary responsibility for each new consultation request the service receives. After obtaining the patient’s case history from the referring provider, the CMR then undertakes a thorough review of the electronic health record and any other available records from other institutions. In the inpatient setting, the CMR performs a new history and physical; phone calls or video conferencing permit history taking for outpatients. In contrast with the standard consultant note, we do not redocument the history, physical, and lab and imaging findings but instead construct a detailed problem list that synthesizes relevant findings into a useful working document.

The service’s faculty leader (BDS) then reviews the problem list with the CMRs to help refine the problem list and begin producing a differential diagnosis during a weekly hour-long meeting. As evidence supports team-based diagnostic collaborations,² the problem list and preliminary differential diagnosis then becomes a shareable document that the CMR or team leader presents to ad hoc general internists, specialists, and the other CMRs. The presentation can be in person, by phone, or e-mail. These ad hoc members, approximately 20 in number and spanning from junior attending physicians to senior clinicians, have volunteered to help the Socrates Project by adding their thoughts on differential diagnoses that explain the problem list and how to move forward with further testing. The ad hoc members have self-identified as clinicians with an interest in medical diagnosis—including surgeons, neurologists, psychiatrists, radiologists, and pathologists—and range in expertise from general internists to subspecialists. Finally, we document our problem list, differential diagnosis, and recommendations in the medical record and discuss the case with the referring team. The service limits its scope of clinical recommendation to diagnosis and avoids commenting on management decisions outside of the use of therapies as empiric diagnostic tests. A sample note is provided as an online Appendix.

Despite our process, we are often left without a satisfying diagnosis. We then are then faced with three possibilities: (1) The diagnosis is identifiable, just not by the physicians involved in the case—we did not think of the diagnosis in our deliberations; (2) The diagnosis is a described condition but without an available test—autoimmune limbic encephalitis associated with an unassayable or unknown auto-antibody, or the acuity of a critically ill patient makes diagnostic testing unreliable or not feasible; (3) The diagnosis has not yet been described by medical science—we are seeing a case of HIV infection in 1971.

With the personnel and process outlined above, we hope to provide recommendations that are useful in guiding a diagnostic workup regardless of which of these three scenarios is applicable. Our flexibility with involving the appropriate specialists in the Socrates Project should minimize the number of patients with a knowable diagnosis that is unknown to us. In the second scenario, our recommendations may rest upon the incorporation of a treatment as a diagnostic test. In the limbic encephalitis example above, a trial of steroids with rapid improvement in the patient’s condition may increase diagnostic certainty. The third scenario is the most difficult to identify. Pattern recognition of similarly presenting patients, keeping ourselves updated on pertinent primary literature, and consideration of advanced diagnostic testing such as exome sequencing and other next-generation sequencing strategies are essential in hoping to characterize a specific clinical syndrome that has yet to be described.

For situations in which our recommendations do not yield a diagnosis, we recognize the role for protocols such as genomic or metagenomic sequencing that assess multiple diagnostic possibilities in parallel without an a priori hypothesis.^3,4 The utility of multi-omics testing in diagnostic workups has been detailed by the Undiagnosed Diseases Network (UDN), which has created a systematic approach to describing new syndromes with the aid of metabolomic and genomic profiling.⁵ It is important to note that even with the resources available to the UDN, the diagnosis rate is 35%, emphasizing that in the majority of diagnosis-refractory cases, a diagnosis will not be found. This low diagnosis rate underscores the need for continued inquiry and cataloging of cases and data for further review or synthesis as the body of medical knowledge continues to expand. For these reasons, we have a follow-up system in place, which involves the assigned CMR regularly reviewing the chart and reporting during our weekly meetings. We make phone calls to patients and providers for cases that appear to be lost to follow-up.

We recognize several important limitations to our care model that may represent barriers to establishing, maintaining, and evaluating a similar service at other institutions. For example, there are limitations and benefits of the CMR as point person for managing our consultations. While they are admittedly junior colleagues with limited experience, CMRs tend to be among the best-read and up-to-date clinicians in the hospital by virtue of their recent general-medicine training and identification as a top clinician and leader. Moreover, in their role with the Socrates Project, CMRs have more time to think, talk with patients, and review the medical record than other clinicians, who may be under pressure to see an increasing number of patients while billing at higher levels. Indeed, the Socrates Project CMRs have, on a number of occasions, been the team members who find the piece of data that no one else thought relevant.

Another factor that may limit establishment of a similar team at other institutions is our volunteer-based model. The Socrates Project members volunteer because they love clinical medicine and serve on the team without remuneration for professional effort. With the CMR role as a notable exception, pressure from achieving relative value unit targets, obtaining grant funding, and publishing primary research publications in their field may limit this care model, particularly when shifting from a clinical-only activity to one that also formally investigates the service’s process and outcomes.

Beyond our clinical objective, we hope that the Socrates Project will further the discovery and description of previously unrecognized disease processes. To that end, we are pursuing an institutional review board-approved protocol to perform a rigorous assessment of the Socrates Project’s process and outcomes, including a cataloging of case archetypes and the time to definitive diagnosis if a diagnosis is established. As we continue to collect data, increasing our referral network may also lead to refinement and improvement in diagnostic processes and outcomes. Over time, we expect that the diagnostic resources available to us will evolve. Utilizing collective intelligence has been shown to improve diagnostic accuracy,⁶ and emerging artificial intelligence technologies may improve diagnostic performance as well.^7,8 Most importantly, through this endeavor, we hope to serve less as an oracle and more as a humble Socratic consultant for clinicians working to reduce diagnostic uncertainty for their patients.

Acknowledgments

The authors wish to thank the Northwestern University Chief Medical Residents, 2015-present, for their tireless efforts in support of the Socrates Project.

Internists are experts in general medicine, skilled at mapping the few hundred ways the human body can go awry onto thousands of diagnoses, and managing the uncertainty inherent in that process. Generalists, almost by definition, consult specialists with their specialty-focused questions; but who does one call for a general consultation about diagnosis if a specific diagnosis remains elusive and the pathology does not fit cleanly into the purview of a consultant? Outside of sage advice from colleagues (usually senior), most medical centers lack a consultation service focused on diagnosis. There is no oracle to seek. In this perspective, we describe our institution’s answer to this problem: the creation of a service for difficult diagnosis based on Socratic principles, particularly the role of iterative hypothesis testing in the process of diagnosis.¹

In 2015, Northwestern Medicine began the Socrates Project, a physician-to-physician consultation service that assists doctors working to diagnose conditions that have so far eluded detection. Our service’s goal is to improve patient care by providing an opinion to the referring physician on diagnostic possibilities for a particular case and ideas to reduce—or at least manage—diagnostic uncertainty.

Most patients referred to the Socrates Project have already undergone an extensive evaluation at top medical centers by experienced clinicians. It would be hubris to assume that we will find a definitive diagnosis in every case; indeed, because of the types of cases referred to our group, it is rare that we find a “Eureka!” diagnosis. When a colleague consults our group, we under-promise in hopes of over-delivering. Instead, we convey to referring physicians that we will conduct a thorough case review and explain our thinking in hopes of uncovering an additional diagnostic avenue, even if that avenue does not ultimately lead to a definitive diagnosis. In addition, the Socrates Project often serves as a broker between consulting services that are deadlocked because of differing diagnostic opinions. We also assist with cases in which a functional disorder is suspected, yet the referring physician is hesitant to diagnose a patient with such a disorder out of concern about missing an important (and possibly obscure) diagnosis.

The Socrates Project receives approximately two consult requests per week, usually from general internists but also from specialists in nearly all disciplines. Around 80% of the referrals are for current inpatients. Our service model is similar to a tumor board, which exists as an interdisciplinary group operating in parallel to the clinical services, to provide consensus-based recommendations. As a result, we act as doctors for doctors, formalizing the curbside consultation. Our usual turnaround time is a week but can be faster for urgent cases. Currently, Socrates Project members, including the faculty leader, volunteer their time and effort at no cost, and there are no charges to patients when physicians consult our group. An overview of the Socrates Project’s personnel and process are outlined in the Figure.

Northwestern’s Chief Medical Residents (CMRs) serve as the fellows for the service, and one of them assumes primary responsibility for each new consultation request the service receives. After obtaining the patient’s case history from the referring provider, the CMR then undertakes a thorough review of the electronic health record and any other available records from other institutions. In the inpatient setting, the CMR performs a new history and physical; phone calls or video conferencing permit history taking for outpatients. In contrast with the standard consultant note, we do not redocument the history, physical, and lab and imaging findings but instead construct a detailed problem list that synthesizes relevant findings into a useful working document.

The service’s faculty leader (BDS) then reviews the problem list with the CMRs to help refine the problem list and begin producing a differential diagnosis during a weekly hour-long meeting. As evidence supports team-based diagnostic collaborations,² the problem list and preliminary differential diagnosis then becomes a shareable document that the CMR or team leader presents to ad hoc general internists, specialists, and the other CMRs. The presentation can be in person, by phone, or e-mail. These ad hoc members, approximately 20 in number and spanning from junior attending physicians to senior clinicians, have volunteered to help the Socrates Project by adding their thoughts on differential diagnoses that explain the problem list and how to move forward with further testing. The ad hoc members have self-identified as clinicians with an interest in medical diagnosis—including surgeons, neurologists, psychiatrists, radiologists, and pathologists—and range in expertise from general internists to subspecialists. Finally, we document our problem list, differential diagnosis, and recommendations in the medical record and discuss the case with the referring team. The service limits its scope of clinical recommendation to diagnosis and avoids commenting on management decisions outside of the use of therapies as empiric diagnostic tests. A sample note is provided as an online Appendix.

Despite our process, we are often left without a satisfying diagnosis. We then are then faced with three possibilities: (1) The diagnosis is identifiable, just not by the physicians involved in the case—we did not think of the diagnosis in our deliberations; (2) The diagnosis is a described condition but without an available test—autoimmune limbic encephalitis associated with an unassayable or unknown auto-antibody, or the acuity of a critically ill patient makes diagnostic testing unreliable or not feasible; (3) The diagnosis has not yet been described by medical science—we are seeing a case of HIV infection in 1971.

With the personnel and process outlined above, we hope to provide recommendations that are useful in guiding a diagnostic workup regardless of which of these three scenarios is applicable. Our flexibility with involving the appropriate specialists in the Socrates Project should minimize the number of patients with a knowable diagnosis that is unknown to us. In the second scenario, our recommendations may rest upon the incorporation of a treatment as a diagnostic test. In the limbic encephalitis example above, a trial of steroids with rapid improvement in the patient’s condition may increase diagnostic certainty. The third scenario is the most difficult to identify. Pattern recognition of similarly presenting patients, keeping ourselves updated on pertinent primary literature, and consideration of advanced diagnostic testing such as exome sequencing and other next-generation sequencing strategies are essential in hoping to characterize a specific clinical syndrome that has yet to be described.

For situations in which our recommendations do not yield a diagnosis, we recognize the role for protocols such as genomic or metagenomic sequencing that assess multiple diagnostic possibilities in parallel without an a priori hypothesis.^3,4 The utility of multi-omics testing in diagnostic workups has been detailed by the Undiagnosed Diseases Network (UDN), which has created a systematic approach to describing new syndromes with the aid of metabolomic and genomic profiling.⁵ It is important to note that even with the resources available to the UDN, the diagnosis rate is 35%, emphasizing that in the majority of diagnosis-refractory cases, a diagnosis will not be found. This low diagnosis rate underscores the need for continued inquiry and cataloging of cases and data for further review or synthesis as the body of medical knowledge continues to expand. For these reasons, we have a follow-up system in place, which involves the assigned CMR regularly reviewing the chart and reporting during our weekly meetings. We make phone calls to patients and providers for cases that appear to be lost to follow-up.

We recognize several important limitations to our care model that may represent barriers to establishing, maintaining, and evaluating a similar service at other institutions. For example, there are limitations and benefits of the CMR as point person for managing our consultations. While they are admittedly junior colleagues with limited experience, CMRs tend to be among the best-read and up-to-date clinicians in the hospital by virtue of their recent general-medicine training and identification as a top clinician and leader. Moreover, in their role with the Socrates Project, CMRs have more time to think, talk with patients, and review the medical record than other clinicians, who may be under pressure to see an increasing number of patients while billing at higher levels. Indeed, the Socrates Project CMRs have, on a number of occasions, been the team members who find the piece of data that no one else thought relevant.

Another factor that may limit establishment of a similar team at other institutions is our volunteer-based model. The Socrates Project members volunteer because they love clinical medicine and serve on the team without remuneration for professional effort. With the CMR role as a notable exception, pressure from achieving relative value unit targets, obtaining grant funding, and publishing primary research publications in their field may limit this care model, particularly when shifting from a clinical-only activity to one that also formally investigates the service’s process and outcomes.

Beyond our clinical objective, we hope that the Socrates Project will further the discovery and description of previously unrecognized disease processes. To that end, we are pursuing an institutional review board-approved protocol to perform a rigorous assessment of the Socrates Project’s process and outcomes, including a cataloging of case archetypes and the time to definitive diagnosis if a diagnosis is established. As we continue to collect data, increasing our referral network may also lead to refinement and improvement in diagnostic processes and outcomes. Over time, we expect that the diagnostic resources available to us will evolve. Utilizing collective intelligence has been shown to improve diagnostic accuracy,⁶ and emerging artificial intelligence technologies may improve diagnostic performance as well.^7,8 Most importantly, through this endeavor, we hope to serve less as an oracle and more as a humble Socratic consultant for clinicians working to reduce diagnostic uncertainty for their patients.

Acknowledgments

The authors wish to thank the Northwestern University Chief Medical Residents, 2015-present, for their tireless efforts in support of the Socrates Project.

Perioperative venous thromboembolism (VTE) is a major contributor to the morbidity and mortality of hospitalized patients. The historical incidence of postoperative VTE varied between 15%-80% depending on the type of procedure and monitoring strategies. Higher incidences of VTE occurred with major surgery (15%-40%), knee or hip arthroplasty (40%-60%), and trauma (60%-80%).¹ The use of VTE prophylaxis with subcutaneous heparin reduced DVTs by 70% and PEs by 50%.² A recent study from Olmstead County showed improved adherence to inpatient VTE prophylaxis from 2005 to 2010 but no difference in VTE incidence. However, 52% of VTE events were associated with hospitalization.³ As such, VTE continues to be a healthcare-associated adverse event, and surgery remains a significant risk factor for thrombosis.⁴

The Agency for Healthcare Research and Quality (AHRQ) released Patient Safety Indicators (PSI) in 2003 to provide a means of screening for adverse events.⁵ Over time, PSIs have been adopted as a measure of hospital performance and are utilized in several pay-for-performance programs. PSI-90 is a composite measure of several other PSIs and is a core metric in the Centers for Medicare and Medicaid Services (CMS) Hospital-Acquired Condition Reduction Program and the Hospital Value-Based Purchasing Program which impacts up to 2% of a hospital’s Medicare payments.^6,7 One component of PSI-90 is PSI-12, which captures perioperative VTE. PSI-12 events are identified using software that screens medical records based on International Classification of Diseases (ICD)-9/10 codes for thrombosis and procedure codes at time of discharge.⁸

Over the last several years, there has been concern regarding the validity of including PSI-12 in pay-for-performance metrics. Common areas for concern include PSI-12’s accuracy in detecting true postoperative VTE⁹ in addition to surveillance bias.^10,11 However, some note that PSI-12 is useful when applied with its original intent: as a screening tool for hospitals to identify specific areas to implement improvements.^9,12The aim of our study was to review all PSI-12 events at our institution to evaluate the accuracy of PSI-12 and identify areas for improvement to prevent VTE events in surgical patients. While several other studies have looked at the positive predictive values, accuracy, or surveillance bias of PSI-12, to the best of our knowledge, few, if any, previous studies have reported PSI-12 events in relation to their timing, type of prophylaxis used, and mitigating factors to identify areas for quality improvement.

PSI-12 events were identified between June 2015 and June 2017 using AHRQ software (version 5). Cases were also identified through Vizient and reviewed to ensure congruence between the methods. Patients’ electronic medical records were reviewed for patient demographics, type of VTE event, platelet count at VTE diagnosis, procedure type, and both the timing and type of VTE prophylaxis. Summary statistics were calculated.

We considered perioperative VTE pharmacologic prophylaxis appropriate if started within 24 hours of a low bleeding risk procedure or 72 hours of a high-bleeding risk procedure.¹³ Mechanical prophylaxis was considered appropriate if pharmacologic prophylaxis was not used because of procedure risk, thrombocytopenia, or active bleeding. The medication administration record was reviewed to determine if prophylaxis was ordered, given, and/or refused.

During the two-year period, 18,084 surgeries were performed, and 161 cases of VTE events were identified. A detailed chart review and correction of documentation led to the exclusion of seven cases (4%) because the VTE event occurred prior to admission (n = 5) or were incidental findings that did not meet the Uniform Hospital Discharge Data Set definition for reporting (n = 2). In total, 154 (0.9% of all surgeries) cases were considered PSI-12 events. Pulmonary embolism (PE) occurred in most cases (n = 97, 62.9%), followed by deep vein thrombosis (DVT) (n = 37, 24.0%). Twenty cases (12.9%) experienced concurrent PE and DVT. Within the PE group, 16 cases (14%) were subsegmental PE only. Eight patients (14% of DVT cases) had only a distal DVT. The mean age of patients was 56 years (+/− 16 years), and the majority (59%) were male. The clinical specialties with the most events included neurosurgery (21%), orthopedics (14%), general surgery (13%), and trauma (11%). Fourteen patients (9%) died during the hospitalization, and of these, six (43%) had either sudden death or death attributed to PE (Table 1).

Cases were also reviewed for the type of VTE prophylactic strategy administered at the time of the event. The top three prophylactic strategies were subcutaneous unfractionated heparin (61%), mechanical prophylaxis only (51%), and enoxaparin (31%). Nine cases of VTE occurred during therapeutic anticoagulation (6%; Table 1).

Utilizing AHRQ version 5 software for PSI-12, our institution identified 154 cases of perioperative VTE. Most of these cases were a pulmonary embolism, occurred within a week of admission, and were associated with surgical specialties that portend a higher risk of VTE. Very few of these cases were deficient in guideline-directed VTE prophylaxis, and several cases had associated factors such as trauma and thrombocytopenia that may have appropriately influenced the decision to use mechanical only prophylaxis. Sixteen percent of pharmacologic VTE prophylactic doses were refused or held for a procedure, a known but rarely quantified influence on rates of pharmacologic prophylaxis. The use of patient-level data and adjudication of the clinical decision that affected the administration of VTE prophylaxis is a major strength of this work. In all, our data raise several questions about the accuracy of PSI-12 in identifying preventable postoperative VTE, especially as it is utilized as a marker for pay-for-performance measures in addition to identifying further areas for research and improvement.

Our results align with previous studies that suggest that PSI-12 is an inaccurate measure of performance quality. A study by Bilimoria et al. in 2013 noted that surveillance biases associated with PSI-12, showing that hospitals with higher compliance to appropriate VTE prophylaxis paradoxically had worse outcomes.¹⁰ Furthermore, Blay et al. recently published a study evaluating PSI-90 scores for hospitals with and without the VTE measure included. Their results indicated that larger hospitals (teaching hospitals, level I trauma centers, etc) caring for sicker patients were noted to improve by 8%-25% when the VTE measure was removed.¹¹ Similarly, our data indicate that the PSI-12 may not be an accurate measure of quality performance, as only 6% of cases were noted to be deficient in appropriate guideline-directed prophylaxis. Even when accounting for the refusal of doses of pharmacologic prophylaxis, this figure only increased to 11%.

Procedures included in the current PSI-12 algorithm also vary in the risk they pose to developing VTE. For example, the current version of PSI-12 includes surgical Medicare Severity Diagnosis Related Groups (MS-DRGs) for procedures with a high risk of VTE such as orthopedic, abdominal, or thoracic procedures.^14,15, However, it is worth noting that procedures such as tracheostomy and ocular surgery are also included.¹⁴ The variation in risk for development of VTE is reflected in the Caprini score, a perioperative risk stratification tool. These latter procedures only contribute one point, whereas trauma would add five points each to the total score, and make the patient a high VTE risk from the procedure alone.¹⁶ With regard to PSI-12, in theory, scores could vary significantly between centers even if the quality of care is the same, based on the volume and risk of procedures performed.

Perioperative venous thromboembolism (VTE) is a major contributor to the morbidity and mortality of hospitalized patients. The historical incidence of postoperative VTE varied between 15%-80% depending on the type of procedure and monitoring strategies. Higher incidences of VTE occurred with major surgery (15%-40%), knee or hip arthroplasty (40%-60%), and trauma (60%-80%).¹ The use of VTE prophylaxis with subcutaneous heparin reduced DVTs by 70% and PEs by 50%.² A recent study from Olmstead County showed improved adherence to inpatient VTE prophylaxis from 2005 to 2010 but no difference in VTE incidence. However, 52% of VTE events were associated with hospitalization.³ As such, VTE continues to be a healthcare-associated adverse event, and surgery remains a significant risk factor for thrombosis.⁴

The Agency for Healthcare Research and Quality (AHRQ) released Patient Safety Indicators (PSI) in 2003 to provide a means of screening for adverse events.⁵ Over time, PSIs have been adopted as a measure of hospital performance and are utilized in several pay-for-performance programs. PSI-90 is a composite measure of several other PSIs and is a core metric in the Centers for Medicare and Medicaid Services (CMS) Hospital-Acquired Condition Reduction Program and the Hospital Value-Based Purchasing Program which impacts up to 2% of a hospital’s Medicare payments.^6,7 One component of PSI-90 is PSI-12, which captures perioperative VTE. PSI-12 events are identified using software that screens medical records based on International Classification of Diseases (ICD)-9/10 codes for thrombosis and procedure codes at time of discharge.⁸

Over the last several years, there has been concern regarding the validity of including PSI-12 in pay-for-performance metrics. Common areas for concern include PSI-12’s accuracy in detecting true postoperative VTE⁹ in addition to surveillance bias.^10,11 However, some note that PSI-12 is useful when applied with its original intent: as a screening tool for hospitals to identify specific areas to implement improvements.^9,12The aim of our study was to review all PSI-12 events at our institution to evaluate the accuracy of PSI-12 and identify areas for improvement to prevent VTE events in surgical patients. While several other studies have looked at the positive predictive values, accuracy, or surveillance bias of PSI-12, to the best of our knowledge, few, if any, previous studies have reported PSI-12 events in relation to their timing, type of prophylaxis used, and mitigating factors to identify areas for quality improvement.

PSI-12 events were identified between June 2015 and June 2017 using AHRQ software (version 5). Cases were also identified through Vizient and reviewed to ensure congruence between the methods. Patients’ electronic medical records were reviewed for patient demographics, type of VTE event, platelet count at VTE diagnosis, procedure type, and both the timing and type of VTE prophylaxis. Summary statistics were calculated.

We considered perioperative VTE pharmacologic prophylaxis appropriate if started within 24 hours of a low bleeding risk procedure or 72 hours of a high-bleeding risk procedure.¹³ Mechanical prophylaxis was considered appropriate if pharmacologic prophylaxis was not used because of procedure risk, thrombocytopenia, or active bleeding. The medication administration record was reviewed to determine if prophylaxis was ordered, given, and/or refused.

During the two-year period, 18,084 surgeries were performed, and 161 cases of VTE events were identified. A detailed chart review and correction of documentation led to the exclusion of seven cases (4%) because the VTE event occurred prior to admission (n = 5) or were incidental findings that did not meet the Uniform Hospital Discharge Data Set definition for reporting (n = 2). In total, 154 (0.9% of all surgeries) cases were considered PSI-12 events. Pulmonary embolism (PE) occurred in most cases (n = 97, 62.9%), followed by deep vein thrombosis (DVT) (n = 37, 24.0%). Twenty cases (12.9%) experienced concurrent PE and DVT. Within the PE group, 16 cases (14%) were subsegmental PE only. Eight patients (14% of DVT cases) had only a distal DVT. The mean age of patients was 56 years (+/− 16 years), and the majority (59%) were male. The clinical specialties with the most events included neurosurgery (21%), orthopedics (14%), general surgery (13%), and trauma (11%). Fourteen patients (9%) died during the hospitalization, and of these, six (43%) had either sudden death or death attributed to PE (Table 1).

Cases were also reviewed for the type of VTE prophylactic strategy administered at the time of the event. The top three prophylactic strategies were subcutaneous unfractionated heparin (61%), mechanical prophylaxis only (51%), and enoxaparin (31%). Nine cases of VTE occurred during therapeutic anticoagulation (6%; Table 1).

Utilizing AHRQ version 5 software for PSI-12, our institution identified 154 cases of perioperative VTE. Most of these cases were a pulmonary embolism, occurred within a week of admission, and were associated with surgical specialties that portend a higher risk of VTE. Very few of these cases were deficient in guideline-directed VTE prophylaxis, and several cases had associated factors such as trauma and thrombocytopenia that may have appropriately influenced the decision to use mechanical only prophylaxis. Sixteen percent of pharmacologic VTE prophylactic doses were refused or held for a procedure, a known but rarely quantified influence on rates of pharmacologic prophylaxis. The use of patient-level data and adjudication of the clinical decision that affected the administration of VTE prophylaxis is a major strength of this work. In all, our data raise several questions about the accuracy of PSI-12 in identifying preventable postoperative VTE, especially as it is utilized as a marker for pay-for-performance measures in addition to identifying further areas for research and improvement.

Our results align with previous studies that suggest that PSI-12 is an inaccurate measure of performance quality. A study by Bilimoria et al. in 2013 noted that surveillance biases associated with PSI-12, showing that hospitals with higher compliance to appropriate VTE prophylaxis paradoxically had worse outcomes.¹⁰ Furthermore, Blay et al. recently published a study evaluating PSI-90 scores for hospitals with and without the VTE measure included. Their results indicated that larger hospitals (teaching hospitals, level I trauma centers, etc) caring for sicker patients were noted to improve by 8%-25% when the VTE measure was removed.¹¹ Similarly, our data indicate that the PSI-12 may not be an accurate measure of quality performance, as only 6% of cases were noted to be deficient in appropriate guideline-directed prophylaxis. Even when accounting for the refusal of doses of pharmacologic prophylaxis, this figure only increased to 11%.

Procedures included in the current PSI-12 algorithm also vary in the risk they pose to developing VTE. For example, the current version of PSI-12 includes surgical Medicare Severity Diagnosis Related Groups (MS-DRGs) for procedures with a high risk of VTE such as orthopedic, abdominal, or thoracic procedures.^14,15, However, it is worth noting that procedures such as tracheostomy and ocular surgery are also included.¹⁴ The variation in risk for development of VTE is reflected in the Caprini score, a perioperative risk stratification tool. These latter procedures only contribute one point, whereas trauma would add five points each to the total score, and make the patient a high VTE risk from the procedure alone.¹⁶ With regard to PSI-12, in theory, scores could vary significantly between centers even if the quality of care is the same, based on the volume and risk of procedures performed.

Perioperative venous thromboembolism (VTE) is a major contributor to the morbidity and mortality of hospitalized patients. The historical incidence of postoperative VTE varied between 15%-80% depending on the type of procedure and monitoring strategies. Higher incidences of VTE occurred with major surgery (15%-40%), knee or hip arthroplasty (40%-60%), and trauma (60%-80%).¹ The use of VTE prophylaxis with subcutaneous heparin reduced DVTs by 70% and PEs by 50%.² A recent study from Olmstead County showed improved adherence to inpatient VTE prophylaxis from 2005 to 2010 but no difference in VTE incidence. However, 52% of VTE events were associated with hospitalization.³ As such, VTE continues to be a healthcare-associated adverse event, and surgery remains a significant risk factor for thrombosis.⁴

The Agency for Healthcare Research and Quality (AHRQ) released Patient Safety Indicators (PSI) in 2003 to provide a means of screening for adverse events.⁵ Over time, PSIs have been adopted as a measure of hospital performance and are utilized in several pay-for-performance programs. PSI-90 is a composite measure of several other PSIs and is a core metric in the Centers for Medicare and Medicaid Services (CMS) Hospital-Acquired Condition Reduction Program and the Hospital Value-Based Purchasing Program which impacts up to 2% of a hospital’s Medicare payments.^6,7 One component of PSI-90 is PSI-12, which captures perioperative VTE. PSI-12 events are identified using software that screens medical records based on International Classification of Diseases (ICD)-9/10 codes for thrombosis and procedure codes at time of discharge.⁸

Over the last several years, there has been concern regarding the validity of including PSI-12 in pay-for-performance metrics. Common areas for concern include PSI-12’s accuracy in detecting true postoperative VTE⁹ in addition to surveillance bias.^10,11 However, some note that PSI-12 is useful when applied with its original intent: as a screening tool for hospitals to identify specific areas to implement improvements.^9,12The aim of our study was to review all PSI-12 events at our institution to evaluate the accuracy of PSI-12 and identify areas for improvement to prevent VTE events in surgical patients. While several other studies have looked at the positive predictive values, accuracy, or surveillance bias of PSI-12, to the best of our knowledge, few, if any, previous studies have reported PSI-12 events in relation to their timing, type of prophylaxis used, and mitigating factors to identify areas for quality improvement.

PSI-12 events were identified between June 2015 and June 2017 using AHRQ software (version 5). Cases were also identified through Vizient and reviewed to ensure congruence between the methods. Patients’ electronic medical records were reviewed for patient demographics, type of VTE event, platelet count at VTE diagnosis, procedure type, and both the timing and type of VTE prophylaxis. Summary statistics were calculated.

We considered perioperative VTE pharmacologic prophylaxis appropriate if started within 24 hours of a low bleeding risk procedure or 72 hours of a high-bleeding risk procedure.¹³ Mechanical prophylaxis was considered appropriate if pharmacologic prophylaxis was not used because of procedure risk, thrombocytopenia, or active bleeding. The medication administration record was reviewed to determine if prophylaxis was ordered, given, and/or refused.

During the two-year period, 18,084 surgeries were performed, and 161 cases of VTE events were identified. A detailed chart review and correction of documentation led to the exclusion of seven cases (4%) because the VTE event occurred prior to admission (n = 5) or were incidental findings that did not meet the Uniform Hospital Discharge Data Set definition for reporting (n = 2). In total, 154 (0.9% of all surgeries) cases were considered PSI-12 events. Pulmonary embolism (PE) occurred in most cases (n = 97, 62.9%), followed by deep vein thrombosis (DVT) (n = 37, 24.0%). Twenty cases (12.9%) experienced concurrent PE and DVT. Within the PE group, 16 cases (14%) were subsegmental PE only. Eight patients (14% of DVT cases) had only a distal DVT. The mean age of patients was 56 years (+/− 16 years), and the majority (59%) were male. The clinical specialties with the most events included neurosurgery (21%), orthopedics (14%), general surgery (13%), and trauma (11%). Fourteen patients (9%) died during the hospitalization, and of these, six (43%) had either sudden death or death attributed to PE (Table 1).

Cases were also reviewed for the type of VTE prophylactic strategy administered at the time of the event. The top three prophylactic strategies were subcutaneous unfractionated heparin (61%), mechanical prophylaxis only (51%), and enoxaparin (31%). Nine cases of VTE occurred during therapeutic anticoagulation (6%; Table 1).

Utilizing AHRQ version 5 software for PSI-12, our institution identified 154 cases of perioperative VTE. Most of these cases were a pulmonary embolism, occurred within a week of admission, and were associated with surgical specialties that portend a higher risk of VTE. Very few of these cases were deficient in guideline-directed VTE prophylaxis, and several cases had associated factors such as trauma and thrombocytopenia that may have appropriately influenced the decision to use mechanical only prophylaxis. Sixteen percent of pharmacologic VTE prophylactic doses were refused or held for a procedure, a known but rarely quantified influence on rates of pharmacologic prophylaxis. The use of patient-level data and adjudication of the clinical decision that affected the administration of VTE prophylaxis is a major strength of this work. In all, our data raise several questions about the accuracy of PSI-12 in identifying preventable postoperative VTE, especially as it is utilized as a marker for pay-for-performance measures in addition to identifying further areas for research and improvement.

Our results align with previous studies that suggest that PSI-12 is an inaccurate measure of performance quality. A study by Bilimoria et al. in 2013 noted that surveillance biases associated with PSI-12, showing that hospitals with higher compliance to appropriate VTE prophylaxis paradoxically had worse outcomes.¹⁰ Furthermore, Blay et al. recently published a study evaluating PSI-90 scores for hospitals with and without the VTE measure included. Their results indicated that larger hospitals (teaching hospitals, level I trauma centers, etc) caring for sicker patients were noted to improve by 8%-25% when the VTE measure was removed.¹¹ Similarly, our data indicate that the PSI-12 may not be an accurate measure of quality performance, as only 6% of cases were noted to be deficient in appropriate guideline-directed prophylaxis. Even when accounting for the refusal of doses of pharmacologic prophylaxis, this figure only increased to 11%.

Procedures included in the current PSI-12 algorithm also vary in the risk they pose to developing VTE. For example, the current version of PSI-12 includes surgical Medicare Severity Diagnosis Related Groups (MS-DRGs) for procedures with a high risk of VTE such as orthopedic, abdominal, or thoracic procedures.^14,15, However, it is worth noting that procedures such as tracheostomy and ocular surgery are also included.¹⁴ The variation in risk for development of VTE is reflected in the Caprini score, a perioperative risk stratification tool. These latter procedures only contribute one point, whereas trauma would add five points each to the total score, and make the patient a high VTE risk from the procedure alone.¹⁶ With regard to PSI-12, in theory, scores could vary significantly between centers even if the quality of care is the same, based on the volume and risk of procedures performed.