Hospital Medicine

A 38-year-old woman presented to her primary care clinic with 3 weeks of progressive numbness and tingling sensation, which began in both hands and then progressed to involve both feet, ascending from the legs to the chest while sparing her buttocks. She also noted weakness of her left leg, but no other motor symptoms were reported. She had no fevers, chills, weight loss, bladder dysfunction, nausea, vomiting, or diarrhea.

As with all neurological complaints, localization of the process will often inform a more specific differential diagnosis. If both sensory and motor findings are present, both central and peripheral nerve processes deserve consideration. The onset of paresthesia in the hands, rapid progression to the trunk, and unilateral leg weakness would be inconsistent with a length-dependent peripheral neuropathy. The distribution of complaints and the sacral sparing suggests a myelopathic process involving the cervical region rather than a cauda equina or conus lesions. In an otherwise healthy person of this age and gender, an inflammatory demyelinating disease affecting the cord including multiple sclerosis (MS) would be a strong consideration, although metabolic, vascular, infectious, compressive, or neoplastic disease of the spinal cord could also present with similar subacute onset and pattern of deficits.

Her medical history included morbid obesity, dry eyes, depression, iron deficiency anemia requiring recurrent intravenous replenishment, and abnormal uterine bleeding. Her surgical history included gastric band placement 7 years earlier with removal 5 years later due to persistent gastroesophageal reflux disease, dysphagia, nausea, and vomiting. The gastric band removal was complicated by chronic abdominal pain. Her medications consisted of duloxetine, intermittent iron infusions, artificial tears, loratadine, and pregabalin. She was sexually active with her husband. She consumed alcohol occasionally but did not smoke tobacco or use illicit drugs.

On exam, her temperature was 36.6°C (97.8°F), blood pressure 132/84 mm Hg, and heart rate 85 beats per minute. Body mass index was 39.5 kg/m². The cardiac, pulmonary, and skin examinations were normal. The abdomen was soft with diffuse tenderness to palpation without rebound or guarding. Examination of cranial nerves 2-12 was normal. Cognition, strength, proprioception, deep tendon reflexes, and light touch were all normal. Her gait was normal, and the Romberg test was negative.

The normal neurologic exam is reassuring but imperfectly sensitive and does not eliminate the possibility of underlying neuropathology. Bariatric surgery may result in an array of nutritional deficiencies such as vitamin E, B12, and copper, which can cause myelopathy and/or neuropathy. However, these abnormalities occur less frequently with gastric banding procedures. If her dry eyes are part of the sicca syndrome, an underlying autoimmune diathesis may be present. Her unexplained chronic abdominal pain prompts considering nonmenstrual causes of iron deficiency anemia, such as celiac disease. Bariatric surgery may contribute to iron deficiency through impaired iron absorption. Her stable weight and lack of diarrhea argue against Crohn’s or celiac disease. Iron deficiency predisposes individuals to pica, most commonly described with ice chip ingestion. If lead pica had occurred, abdominal and neurological symptoms could result. Nevertheless, the abdominal pain is nonspecific, and its occurrence after gastric band removal makes its link to her neurologic syndrome unclear. An initial evaluation would include basic metabolic panel, complete blood count with differential, erythrocyte sedimentation rate, C-reactive protein (CRP), thyroid-stimulating hormone, vitamin B12, and copper levels.

A basic metabolic panel was normal. The white cell count was 5,710 per cubic millimeter, hemoglobin level 12.2 g per deciliter, mean corpuscular volume 85.2 fl, and platelet count 279,000 per cubic millimeter. The serum ferritin level was 18 ng per milliliter (normal range, 13-150), iron 28 µg per deciliter (normal range, 50-170), total iron-binding capacity 364 µg per deciliter (normal range, 250-450), and iron saturation 8% (normal range, 20-55). The vitamin B12 level was 621 pg per milliliter (normal range, 232-1,245) and thyroid-stimulating hormone level 1.87 units per milliliter (normal range, 0.50-4.50). Electrolyte and aminotransferase levels were within normal limits. CRP was 1.0 mg per deciliter (normal range, <0.5) and erythrocyte sedimentation rate 33 millimeters per hour (normal range, 4-25). Hepatitis C and HIV antibodies were nonreactive.

The ongoing iron deficiency despite parenteral iron replacement raises the question of ongoing gastrointestinal or genitourinary blood loss. While the level of vitamin B12 in the serum may be misleadingly normal with cobalamin deficiency, a methylmalonic acid level is indicated to evaluate whether tissue stores are depleted. Copper levels are warranted given the prior bariatric surgery. The mild elevations of inflammatory markers are nonspecific but reduce the likelihood of a highly inflammatory process to account for the neurological and abdominal symptoms. 

At her 3-month follow-up visit, she noted that the paresthesia had improved and was now limited to her bilateral lower extremities. During the same clinic visit, she experienced a 45-minute episode of ascending left upper extremity numbness. Her physical examination revealed normal strength and reflexes. She had diminished response to pinprick in both legs to the knees and in both hands to the wrists. Vibration sense was diminished in the bilateral lower extremities.

A glycosylated hemoglobin (HbA1c) level was 6.2%. Methylmalonic acid was 69 nmol per liter (normal range, 45-325). Antibodies to Borrelia burgdorferi and Treponema pallidum were absent. Impaired glucose metabolism was the leading diagnosis for her polyneuropathy, and it was recommended that she undergo an oral glucose tolerance test. Electromyography was not performed.

The neurological symptoms are now chronic, and importantly, the patient has developed sensory deficits on neurological examination, suggesting worsening of the underlying process. While the paresthesia is now limited to a “stocking/glove” distribution consistent with distal sensory polyneuropathy, there should still be a concern for spinal cord pathology given that the HbA1c level of 6.2 would not explain her initial distribution of symptoms. Myelopathy may mimic peripheral nerve disease if, for example, there is involvement of the dorsal columns leading to sensory deficits of vibration and proprioception. Additionally, the transient episode of upper extremity numbness raises the question of sensory nerve root involvement (ie, sensory radiculopathy). Unexplained abdominal pain could possibly represent the involvement of other nerve roots innervating the abdominal wall. The patient’s episode of focal arm numbness recalls the lancinating radicular pain of tabes dorsalis; however, the negative specific treponemal antibody test excludes neurosyphilis.

The differential diagnosis going forward will be strongly conditioned by the localization of the neurological lesion(s). To differentiate between myelopathy, radiculopathy, and peripheral neuropathy, I would perform nerve conduction studies, magnetic resonance imaging (MRI) of the spinal cord, and cerebrospinal fluid analysis.

The patient began taking a multivitamin, and after weeks her paresthesia had resolved. One month later, she developed an intermittent, throbbing left-sided headache and pain behind the left eye that was worsened with ocular movement. She then noted decreased visual acuity in her left eye that progressed the following month. She denied photophobia, flashers, or floaters.

In the emergency department, visual acuity was 20/25 in her right eye; in the left eye she was only able to count fingers. Extraocular movements of both eyes were normal as was her right pupillary reflex. Red desaturation and a relative afferent papillary defect were present in the left eye. Fundoscopic exam demonstrated left optic disc swelling. The remainder of her cranial nerves were normal. She had pronation of the left upper extremity and mild right finger-to-nose dysmetria. Muscle tone, strength, sensation, and deep tendon reflexes were normal.

The improvement in the sensory symptoms was unlikely to be related to the nutritional intervention and provides a clue to an underlying waxing and waning illness. That interpretation is supported by the subsequent development of new visual symptoms and signs, which point to optic nerve pathology. Optic neuropathy has a broad differential diagnosis that includes ischemic, metabolic, toxic, and compressive causes. Eye pain, swelling of the optic disc, and prominent impairment of color vision all point to the more specific syndrome of optic neuritis caused by infections (including both Treponema pallidum and Borrelia species), systemic autoimmune diseases (systemic lupus erythematosus or Sjogren’s syndrome), and central nervous system (CNS) demyelinating diseases. Of these, inflammatory demyelinating processes would be the likeliest explanation of intermittent and improving neurologic findings.

With relapsing symptoms and findings that are separate in distribution and time, two diagnoses become most likely, and both of these are most often diagnosed in young women. MS is common, and optic neuritis occurs in more than 50% of patients over the course of illness. Neuromyelitis optica spectrum disorder (NMOSD) is a rare condition that can exist in isolation or be associated with other autoimmune illnesses. While these entities are difficult to differentiate clinically, neuroimaging that demonstrates extensive intracerebral demyelinating lesions and cerebrospinal fluid with oligoclonal bands favor MS, whereas extensive, predominant spinal cord involvement is suggestive of NMOSD. Approximately 70% of NMO patients harbor an antibody directed against the aquaporin-4 channel, and these antibodies are not seen in patients with MS. A milder NMO-like disorder has also been associated with antimyelin oligodendrocyte antibodies (MOG).

Testing for antinuclear antibodies, anti–double-stranded DNA, anti-Ro (SSA), and anti-La (SSB) antibodies was negative. The level of C3 was 162 mg per deciliter (normal range 81-157) and C4 38 (normal range 13-39). T-spot testing for latent tuberculosis was negative.

There is no serological evidence of active systemic lupus erythematosus or Sjogren’s syndrome. The pretest probability of CNS tuberculosis was low in light of her presenting complaints, relatively protracted course, and overall clinical stability without antituberculous therapy. Tests for latent tuberculosis infection have significant limitations of both sensitivity and specificity for the diagnosis of active disease.

Optical coherence tomography showed optic disc edema in the left eye only. MRI of the head with contrast revealed abnormal signal intensity involving the posterior aspect of the pons, right middle cerebellar peduncle, anterior left temporal lobe, bilateral periventricular white matter, subcortical white matter of the frontal lobes bilaterally, and medulla with abnormal signal and enhancement of the left optic nerve (Figure, Panel A). MRI of the cervical and thoracic spine demonstrated multifocal demyelinating lesions at C3, C4, C7, T4, T5, T7, and T8 (Figure, Panel B). The lesions were not longitudinally extensive. There was no significant postcontrast enhancement to suggest active demyelination.

The cerebrospinal fluid analysis revealed glucose of 105 mg per deciliter and a total protein of 26.1 mg per deciliter. In the fourth tube, there were 20 red cells per cubic and four white cells with a differential of 62% neutrophils, 35% lymphocytes, and 3% monocytes. Epstein-Barr and herpes simplex virus DNA were negative. A Venereal Disease Research Laboratory test was negative. Multiple oligoclonal IgG bands were identified only in the cerebrospinal fluid. Aquaporin-4 IgG and MOG antibodies were negative.

In addition to the expected finding of enhancement of the optic nerve, MRI demonstrated numerous multifocal white matter lesions throughout the cerebrum, brainstem, and spinal cord. Many of the lesions were in “silent” areas, which is not directly attributable to specific symptoms, but several did correlate with the subtler deficits of weakness and dysmetria that were noted on examination. Although such lesions may be seen with a diverse group of systemic diseases including adrenal leukodystrophy, sarcoidosis, Behcet’s, cerebral lupus, and vasculitis, primary CNS inflammatory demyelinating diseases are much more likely. The extensive distribution of demyelination argues against NMOSD. The negative aquaporin-4 and MOG assays support this conclusion. Not all multifocal CNS demyelination is caused by MS and can be seen in posterior reversible encephalopathy syndrome, cerebral autosomal dominant arteriopathy with subcortical infarcts and leukoencephalopathy, and adult polyglucosan body disease. Osmotic demyelination is increasingly being recognized as a process that can be more widespread rather than just being limited to the pons. Viral infections of the CNS such as the JC virus (PML) may also provoke multifocal demyelination. Acute disseminated encephalomyelitis is most often seen during childhood, usually after vaccination or after an infectious prodrome. The tempo of the progression of these other diseases tends to be much more rapid than this woman’s course, and often, the neurological deficits are more profound and debilitating. The clinical presentation of sensory-predominant myelopathy, followed by optic neuritis, absence of systemic inflammatory signs or laboratory markers, exclusion of other relevant diseases, multifocal white matter lesions on imaging, minimal pleocytosis, and presence of oligoclonal bands in cerebrospinal fluid, all point to a diagnosis of relapsing-remitting MS.

The patient was diagnosed with MS. She was admitted to the neurology service and treated with 1,000 mg IV methylprednisolone for 3 days with a prompt improvement in her vision. She was started on natalizumab without a relapse of symptoms over the past year.

Multiple sclerosis is a chronic demyelinating disease of the CNS.¹ The diagnosis of MS has classically been based upon compatible clinical and radiographic evidence of pathology that is disseminated in space and time. Patients typically present with an initial clinically isolated syndrome—involving changes in vision, sensation, strength, mobility, or cognition—for which there is radiographic evidence of demyelination.² A diagnosis of clinically definite MS is then often made based on a subsequent relapse of symptoms.³

An interval from initial symptoms has been central to the diagnosis of MS (“lesions disseminated in time”). However, recent evidence questions this diagnostic paradigm, and a more rapid diagnosis of MS has been recommended. This recommendation is reflected in the updated McDonald criteria, according to which, if a clinical presentation is supported by the presence of oligoclonal bands in the cerebrospinal fluid, a diagnosis can be made on the basis of radiographic evidence of dissemination of disease in space, without evidence of dissemination in time.⁴ The importance of such early diagnosis has been supported by numerous studies that have demonstrated improved clinical outcomes with early therapy.^5-7

Despite the McDonald criteria, delays in definitive diagnosis are common in MS. Patients with MS in Spain were found to experience a 2-year delay from the first onset of symptoms to diagnosis.⁸ In this cohort, patients exhibited delays in presenting to a healthcare provider, as well as delays in diagnosis with an average time from seeing an initial provider to diagnosis of 6 months. When patients who were referred for a demyelinating episode were surveyed, over a third reported a prior suggestive event.⁹ The time from the first suggestive episode to referral to a neurologist for a recognized demyelinating event was 46 months. Other studies have shown that delays in diagnosis are especially common in younger patients, those with primary progressive MS, and those with comorbid disease.^10,11

Misapplication of an MS diagnosis also occurs frequently. In one case series, such misapplication was found most often in cases involving migraine, fibromyalgia, psychogenic disorders, and NMOSD.¹² NMOSD is distinguished from MS by the presence of typical brain and spine findings on MRI.¹³ Antibodies to aquaporin-4 are highly specific and moderately sensitive for the disease.¹⁴ It is important to distinguish NMOSD from MS as certain disease-modifying drugs used for MS might actually exacerbate NMOSD.¹⁵ A lesion that traverses over three or more contiguous vertebral segments with predominant involvement of central gray matter (ie, longitudinally extensive transverse myelitis) on MRI is the most distinct finding of NMOSD. In contrast, similar to our patient, short and often multiple lesions are demonstrated on spinal cord MRI in patients with MS. Sensitive and specific findings of brain MRI in patients with MS include the presence of lateral ventricle and inferior temporal lobe lesion, Dawson’s fingers, central vein sign, or an S-shaped U-fiber lesion. In NMOSD, brain MRI might reveal periependymal lesions surrounding the ventricular system.

This case highlights the diagnostic challenges related to presentations of a waxing and waning neurological process. At the time of the second evaluation, the presentation was interpreted as a length-dependent polyneuropathy due to glucose intolerance. Our patient’s relatively normal HbA1c, subacute onset of neuropathic symptoms (ie, <4 weeks), sensory and motor complaints, and onset in the upper extremities suggested an alternative diagnosis to prediabetes. Once the patient presented with optic neuritis, the cause of the initial symptoms was obvious, but then, hindsight is 20/20.

• Early treatment of MS results in improved clinical outcomes.
• Delays in the definitive diagnosis of MS are common, especially in younger patients, those with primary progressive MS, and those with comorbid disease.
• If a clinical presentation is supported by the presence of oligoclonal bands in the cerebrospinal fluid, a diagnosis of MS can be made on the basis of radiographic evidence of dissemination of disease in space, without evidence of dissemination in time.

Acknowledgments

The authors wish to thank Rabih Geha, MD, and Gurpreet Dhaliwal, MD, for providing feedback on an earlier version of this manuscript.

A 38-year-old woman presented to her primary care clinic with 3 weeks of progressive numbness and tingling sensation, which began in both hands and then progressed to involve both feet, ascending from the legs to the chest while sparing her buttocks. She also noted weakness of her left leg, but no other motor symptoms were reported. She had no fevers, chills, weight loss, bladder dysfunction, nausea, vomiting, or diarrhea.

As with all neurological complaints, localization of the process will often inform a more specific differential diagnosis. If both sensory and motor findings are present, both central and peripheral nerve processes deserve consideration. The onset of paresthesia in the hands, rapid progression to the trunk, and unilateral leg weakness would be inconsistent with a length-dependent peripheral neuropathy. The distribution of complaints and the sacral sparing suggests a myelopathic process involving the cervical region rather than a cauda equina or conus lesions. In an otherwise healthy person of this age and gender, an inflammatory demyelinating disease affecting the cord including multiple sclerosis (MS) would be a strong consideration, although metabolic, vascular, infectious, compressive, or neoplastic disease of the spinal cord could also present with similar subacute onset and pattern of deficits.

Her medical history included morbid obesity, dry eyes, depression, iron deficiency anemia requiring recurrent intravenous replenishment, and abnormal uterine bleeding. Her surgical history included gastric band placement 7 years earlier with removal 5 years later due to persistent gastroesophageal reflux disease, dysphagia, nausea, and vomiting. The gastric band removal was complicated by chronic abdominal pain. Her medications consisted of duloxetine, intermittent iron infusions, artificial tears, loratadine, and pregabalin. She was sexually active with her husband. She consumed alcohol occasionally but did not smoke tobacco or use illicit drugs.

On exam, her temperature was 36.6°C (97.8°F), blood pressure 132/84 mm Hg, and heart rate 85 beats per minute. Body mass index was 39.5 kg/m². The cardiac, pulmonary, and skin examinations were normal. The abdomen was soft with diffuse tenderness to palpation without rebound or guarding. Examination of cranial nerves 2-12 was normal. Cognition, strength, proprioception, deep tendon reflexes, and light touch were all normal. Her gait was normal, and the Romberg test was negative.

The normal neurologic exam is reassuring but imperfectly sensitive and does not eliminate the possibility of underlying neuropathology. Bariatric surgery may result in an array of nutritional deficiencies such as vitamin E, B12, and copper, which can cause myelopathy and/or neuropathy. However, these abnormalities occur less frequently with gastric banding procedures. If her dry eyes are part of the sicca syndrome, an underlying autoimmune diathesis may be present. Her unexplained chronic abdominal pain prompts considering nonmenstrual causes of iron deficiency anemia, such as celiac disease. Bariatric surgery may contribute to iron deficiency through impaired iron absorption. Her stable weight and lack of diarrhea argue against Crohn’s or celiac disease. Iron deficiency predisposes individuals to pica, most commonly described with ice chip ingestion. If lead pica had occurred, abdominal and neurological symptoms could result. Nevertheless, the abdominal pain is nonspecific, and its occurrence after gastric band removal makes its link to her neurologic syndrome unclear. An initial evaluation would include basic metabolic panel, complete blood count with differential, erythrocyte sedimentation rate, C-reactive protein (CRP), thyroid-stimulating hormone, vitamin B12, and copper levels.

A basic metabolic panel was normal. The white cell count was 5,710 per cubic millimeter, hemoglobin level 12.2 g per deciliter, mean corpuscular volume 85.2 fl, and platelet count 279,000 per cubic millimeter. The serum ferritin level was 18 ng per milliliter (normal range, 13-150), iron 28 µg per deciliter (normal range, 50-170), total iron-binding capacity 364 µg per deciliter (normal range, 250-450), and iron saturation 8% (normal range, 20-55). The vitamin B12 level was 621 pg per milliliter (normal range, 232-1,245) and thyroid-stimulating hormone level 1.87 units per milliliter (normal range, 0.50-4.50). Electrolyte and aminotransferase levels were within normal limits. CRP was 1.0 mg per deciliter (normal range, <0.5) and erythrocyte sedimentation rate 33 millimeters per hour (normal range, 4-25). Hepatitis C and HIV antibodies were nonreactive.

The ongoing iron deficiency despite parenteral iron replacement raises the question of ongoing gastrointestinal or genitourinary blood loss. While the level of vitamin B12 in the serum may be misleadingly normal with cobalamin deficiency, a methylmalonic acid level is indicated to evaluate whether tissue stores are depleted. Copper levels are warranted given the prior bariatric surgery. The mild elevations of inflammatory markers are nonspecific but reduce the likelihood of a highly inflammatory process to account for the neurological and abdominal symptoms. 

At her 3-month follow-up visit, she noted that the paresthesia had improved and was now limited to her bilateral lower extremities. During the same clinic visit, she experienced a 45-minute episode of ascending left upper extremity numbness. Her physical examination revealed normal strength and reflexes. She had diminished response to pinprick in both legs to the knees and in both hands to the wrists. Vibration sense was diminished in the bilateral lower extremities.

A glycosylated hemoglobin (HbA1c) level was 6.2%. Methylmalonic acid was 69 nmol per liter (normal range, 45-325). Antibodies to Borrelia burgdorferi and Treponema pallidum were absent. Impaired glucose metabolism was the leading diagnosis for her polyneuropathy, and it was recommended that she undergo an oral glucose tolerance test. Electromyography was not performed.

The neurological symptoms are now chronic, and importantly, the patient has developed sensory deficits on neurological examination, suggesting worsening of the underlying process. While the paresthesia is now limited to a “stocking/glove” distribution consistent with distal sensory polyneuropathy, there should still be a concern for spinal cord pathology given that the HbA1c level of 6.2 would not explain her initial distribution of symptoms. Myelopathy may mimic peripheral nerve disease if, for example, there is involvement of the dorsal columns leading to sensory deficits of vibration and proprioception. Additionally, the transient episode of upper extremity numbness raises the question of sensory nerve root involvement (ie, sensory radiculopathy). Unexplained abdominal pain could possibly represent the involvement of other nerve roots innervating the abdominal wall. The patient’s episode of focal arm numbness recalls the lancinating radicular pain of tabes dorsalis; however, the negative specific treponemal antibody test excludes neurosyphilis.

The differential diagnosis going forward will be strongly conditioned by the localization of the neurological lesion(s). To differentiate between myelopathy, radiculopathy, and peripheral neuropathy, I would perform nerve conduction studies, magnetic resonance imaging (MRI) of the spinal cord, and cerebrospinal fluid analysis.

The patient began taking a multivitamin, and after weeks her paresthesia had resolved. One month later, she developed an intermittent, throbbing left-sided headache and pain behind the left eye that was worsened with ocular movement. She then noted decreased visual acuity in her left eye that progressed the following month. She denied photophobia, flashers, or floaters.

In the emergency department, visual acuity was 20/25 in her right eye; in the left eye she was only able to count fingers. Extraocular movements of both eyes were normal as was her right pupillary reflex. Red desaturation and a relative afferent papillary defect were present in the left eye. Fundoscopic exam demonstrated left optic disc swelling. The remainder of her cranial nerves were normal. She had pronation of the left upper extremity and mild right finger-to-nose dysmetria. Muscle tone, strength, sensation, and deep tendon reflexes were normal.

The improvement in the sensory symptoms was unlikely to be related to the nutritional intervention and provides a clue to an underlying waxing and waning illness. That interpretation is supported by the subsequent development of new visual symptoms and signs, which point to optic nerve pathology. Optic neuropathy has a broad differential diagnosis that includes ischemic, metabolic, toxic, and compressive causes. Eye pain, swelling of the optic disc, and prominent impairment of color vision all point to the more specific syndrome of optic neuritis caused by infections (including both Treponema pallidum and Borrelia species), systemic autoimmune diseases (systemic lupus erythematosus or Sjogren’s syndrome), and central nervous system (CNS) demyelinating diseases. Of these, inflammatory demyelinating processes would be the likeliest explanation of intermittent and improving neurologic findings.

With relapsing symptoms and findings that are separate in distribution and time, two diagnoses become most likely, and both of these are most often diagnosed in young women. MS is common, and optic neuritis occurs in more than 50% of patients over the course of illness. Neuromyelitis optica spectrum disorder (NMOSD) is a rare condition that can exist in isolation or be associated with other autoimmune illnesses. While these entities are difficult to differentiate clinically, neuroimaging that demonstrates extensive intracerebral demyelinating lesions and cerebrospinal fluid with oligoclonal bands favor MS, whereas extensive, predominant spinal cord involvement is suggestive of NMOSD. Approximately 70% of NMO patients harbor an antibody directed against the aquaporin-4 channel, and these antibodies are not seen in patients with MS. A milder NMO-like disorder has also been associated with antimyelin oligodendrocyte antibodies (MOG).

Testing for antinuclear antibodies, anti–double-stranded DNA, anti-Ro (SSA), and anti-La (SSB) antibodies was negative. The level of C3 was 162 mg per deciliter (normal range 81-157) and C4 38 (normal range 13-39). T-spot testing for latent tuberculosis was negative.

There is no serological evidence of active systemic lupus erythematosus or Sjogren’s syndrome. The pretest probability of CNS tuberculosis was low in light of her presenting complaints, relatively protracted course, and overall clinical stability without antituberculous therapy. Tests for latent tuberculosis infection have significant limitations of both sensitivity and specificity for the diagnosis of active disease.

Optical coherence tomography showed optic disc edema in the left eye only. MRI of the head with contrast revealed abnormal signal intensity involving the posterior aspect of the pons, right middle cerebellar peduncle, anterior left temporal lobe, bilateral periventricular white matter, subcortical white matter of the frontal lobes bilaterally, and medulla with abnormal signal and enhancement of the left optic nerve (Figure, Panel A). MRI of the cervical and thoracic spine demonstrated multifocal demyelinating lesions at C3, C4, C7, T4, T5, T7, and T8 (Figure, Panel B). The lesions were not longitudinally extensive. There was no significant postcontrast enhancement to suggest active demyelination.

The cerebrospinal fluid analysis revealed glucose of 105 mg per deciliter and a total protein of 26.1 mg per deciliter. In the fourth tube, there were 20 red cells per cubic and four white cells with a differential of 62% neutrophils, 35% lymphocytes, and 3% monocytes. Epstein-Barr and herpes simplex virus DNA were negative. A Venereal Disease Research Laboratory test was negative. Multiple oligoclonal IgG bands were identified only in the cerebrospinal fluid. Aquaporin-4 IgG and MOG antibodies were negative.

In addition to the expected finding of enhancement of the optic nerve, MRI demonstrated numerous multifocal white matter lesions throughout the cerebrum, brainstem, and spinal cord. Many of the lesions were in “silent” areas, which is not directly attributable to specific symptoms, but several did correlate with the subtler deficits of weakness and dysmetria that were noted on examination. Although such lesions may be seen with a diverse group of systemic diseases including adrenal leukodystrophy, sarcoidosis, Behcet’s, cerebral lupus, and vasculitis, primary CNS inflammatory demyelinating diseases are much more likely. The extensive distribution of demyelination argues against NMOSD. The negative aquaporin-4 and MOG assays support this conclusion. Not all multifocal CNS demyelination is caused by MS and can be seen in posterior reversible encephalopathy syndrome, cerebral autosomal dominant arteriopathy with subcortical infarcts and leukoencephalopathy, and adult polyglucosan body disease. Osmotic demyelination is increasingly being recognized as a process that can be more widespread rather than just being limited to the pons. Viral infections of the CNS such as the JC virus (PML) may also provoke multifocal demyelination. Acute disseminated encephalomyelitis is most often seen during childhood, usually after vaccination or after an infectious prodrome. The tempo of the progression of these other diseases tends to be much more rapid than this woman’s course, and often, the neurological deficits are more profound and debilitating. The clinical presentation of sensory-predominant myelopathy, followed by optic neuritis, absence of systemic inflammatory signs or laboratory markers, exclusion of other relevant diseases, multifocal white matter lesions on imaging, minimal pleocytosis, and presence of oligoclonal bands in cerebrospinal fluid, all point to a diagnosis of relapsing-remitting MS.

The patient was diagnosed with MS. She was admitted to the neurology service and treated with 1,000 mg IV methylprednisolone for 3 days with a prompt improvement in her vision. She was started on natalizumab without a relapse of symptoms over the past year.

Multiple sclerosis is a chronic demyelinating disease of the CNS.¹ The diagnosis of MS has classically been based upon compatible clinical and radiographic evidence of pathology that is disseminated in space and time. Patients typically present with an initial clinically isolated syndrome—involving changes in vision, sensation, strength, mobility, or cognition—for which there is radiographic evidence of demyelination.² A diagnosis of clinically definite MS is then often made based on a subsequent relapse of symptoms.³

An interval from initial symptoms has been central to the diagnosis of MS (“lesions disseminated in time”). However, recent evidence questions this diagnostic paradigm, and a more rapid diagnosis of MS has been recommended. This recommendation is reflected in the updated McDonald criteria, according to which, if a clinical presentation is supported by the presence of oligoclonal bands in the cerebrospinal fluid, a diagnosis can be made on the basis of radiographic evidence of dissemination of disease in space, without evidence of dissemination in time.⁴ The importance of such early diagnosis has been supported by numerous studies that have demonstrated improved clinical outcomes with early therapy.^5-7

Despite the McDonald criteria, delays in definitive diagnosis are common in MS. Patients with MS in Spain were found to experience a 2-year delay from the first onset of symptoms to diagnosis.⁸ In this cohort, patients exhibited delays in presenting to a healthcare provider, as well as delays in diagnosis with an average time from seeing an initial provider to diagnosis of 6 months. When patients who were referred for a demyelinating episode were surveyed, over a third reported a prior suggestive event.⁹ The time from the first suggestive episode to referral to a neurologist for a recognized demyelinating event was 46 months. Other studies have shown that delays in diagnosis are especially common in younger patients, those with primary progressive MS, and those with comorbid disease.^10,11

Misapplication of an MS diagnosis also occurs frequently. In one case series, such misapplication was found most often in cases involving migraine, fibromyalgia, psychogenic disorders, and NMOSD.¹² NMOSD is distinguished from MS by the presence of typical brain and spine findings on MRI.¹³ Antibodies to aquaporin-4 are highly specific and moderately sensitive for the disease.¹⁴ It is important to distinguish NMOSD from MS as certain disease-modifying drugs used for MS might actually exacerbate NMOSD.¹⁵ A lesion that traverses over three or more contiguous vertebral segments with predominant involvement of central gray matter (ie, longitudinally extensive transverse myelitis) on MRI is the most distinct finding of NMOSD. In contrast, similar to our patient, short and often multiple lesions are demonstrated on spinal cord MRI in patients with MS. Sensitive and specific findings of brain MRI in patients with MS include the presence of lateral ventricle and inferior temporal lobe lesion, Dawson’s fingers, central vein sign, or an S-shaped U-fiber lesion. In NMOSD, brain MRI might reveal periependymal lesions surrounding the ventricular system.

This case highlights the diagnostic challenges related to presentations of a waxing and waning neurological process. At the time of the second evaluation, the presentation was interpreted as a length-dependent polyneuropathy due to glucose intolerance. Our patient’s relatively normal HbA1c, subacute onset of neuropathic symptoms (ie, <4 weeks), sensory and motor complaints, and onset in the upper extremities suggested an alternative diagnosis to prediabetes. Once the patient presented with optic neuritis, the cause of the initial symptoms was obvious, but then, hindsight is 20/20.

• Early treatment of MS results in improved clinical outcomes.
• Delays in the definitive diagnosis of MS are common, especially in younger patients, those with primary progressive MS, and those with comorbid disease.
• If a clinical presentation is supported by the presence of oligoclonal bands in the cerebrospinal fluid, a diagnosis of MS can be made on the basis of radiographic evidence of dissemination of disease in space, without evidence of dissemination in time.

Acknowledgments

The authors wish to thank Rabih Geha, MD, and Gurpreet Dhaliwal, MD, for providing feedback on an earlier version of this manuscript.

A 38-year-old woman presented to her primary care clinic with 3 weeks of progressive numbness and tingling sensation, which began in both hands and then progressed to involve both feet, ascending from the legs to the chest while sparing her buttocks. She also noted weakness of her left leg, but no other motor symptoms were reported. She had no fevers, chills, weight loss, bladder dysfunction, nausea, vomiting, or diarrhea.

As with all neurological complaints, localization of the process will often inform a more specific differential diagnosis. If both sensory and motor findings are present, both central and peripheral nerve processes deserve consideration. The onset of paresthesia in the hands, rapid progression to the trunk, and unilateral leg weakness would be inconsistent with a length-dependent peripheral neuropathy. The distribution of complaints and the sacral sparing suggests a myelopathic process involving the cervical region rather than a cauda equina or conus lesions. In an otherwise healthy person of this age and gender, an inflammatory demyelinating disease affecting the cord including multiple sclerosis (MS) would be a strong consideration, although metabolic, vascular, infectious, compressive, or neoplastic disease of the spinal cord could also present with similar subacute onset and pattern of deficits.

Her medical history included morbid obesity, dry eyes, depression, iron deficiency anemia requiring recurrent intravenous replenishment, and abnormal uterine bleeding. Her surgical history included gastric band placement 7 years earlier with removal 5 years later due to persistent gastroesophageal reflux disease, dysphagia, nausea, and vomiting. The gastric band removal was complicated by chronic abdominal pain. Her medications consisted of duloxetine, intermittent iron infusions, artificial tears, loratadine, and pregabalin. She was sexually active with her husband. She consumed alcohol occasionally but did not smoke tobacco or use illicit drugs.

On exam, her temperature was 36.6°C (97.8°F), blood pressure 132/84 mm Hg, and heart rate 85 beats per minute. Body mass index was 39.5 kg/m². The cardiac, pulmonary, and skin examinations were normal. The abdomen was soft with diffuse tenderness to palpation without rebound or guarding. Examination of cranial nerves 2-12 was normal. Cognition, strength, proprioception, deep tendon reflexes, and light touch were all normal. Her gait was normal, and the Romberg test was negative.

The normal neurologic exam is reassuring but imperfectly sensitive and does not eliminate the possibility of underlying neuropathology. Bariatric surgery may result in an array of nutritional deficiencies such as vitamin E, B12, and copper, which can cause myelopathy and/or neuropathy. However, these abnormalities occur less frequently with gastric banding procedures. If her dry eyes are part of the sicca syndrome, an underlying autoimmune diathesis may be present. Her unexplained chronic abdominal pain prompts considering nonmenstrual causes of iron deficiency anemia, such as celiac disease. Bariatric surgery may contribute to iron deficiency through impaired iron absorption. Her stable weight and lack of diarrhea argue against Crohn’s or celiac disease. Iron deficiency predisposes individuals to pica, most commonly described with ice chip ingestion. If lead pica had occurred, abdominal and neurological symptoms could result. Nevertheless, the abdominal pain is nonspecific, and its occurrence after gastric band removal makes its link to her neurologic syndrome unclear. An initial evaluation would include basic metabolic panel, complete blood count with differential, erythrocyte sedimentation rate, C-reactive protein (CRP), thyroid-stimulating hormone, vitamin B12, and copper levels.

A basic metabolic panel was normal. The white cell count was 5,710 per cubic millimeter, hemoglobin level 12.2 g per deciliter, mean corpuscular volume 85.2 fl, and platelet count 279,000 per cubic millimeter. The serum ferritin level was 18 ng per milliliter (normal range, 13-150), iron 28 µg per deciliter (normal range, 50-170), total iron-binding capacity 364 µg per deciliter (normal range, 250-450), and iron saturation 8% (normal range, 20-55). The vitamin B12 level was 621 pg per milliliter (normal range, 232-1,245) and thyroid-stimulating hormone level 1.87 units per milliliter (normal range, 0.50-4.50). Electrolyte and aminotransferase levels were within normal limits. CRP was 1.0 mg per deciliter (normal range, <0.5) and erythrocyte sedimentation rate 33 millimeters per hour (normal range, 4-25). Hepatitis C and HIV antibodies were nonreactive.

The ongoing iron deficiency despite parenteral iron replacement raises the question of ongoing gastrointestinal or genitourinary blood loss. While the level of vitamin B12 in the serum may be misleadingly normal with cobalamin deficiency, a methylmalonic acid level is indicated to evaluate whether tissue stores are depleted. Copper levels are warranted given the prior bariatric surgery. The mild elevations of inflammatory markers are nonspecific but reduce the likelihood of a highly inflammatory process to account for the neurological and abdominal symptoms. 

At her 3-month follow-up visit, she noted that the paresthesia had improved and was now limited to her bilateral lower extremities. During the same clinic visit, she experienced a 45-minute episode of ascending left upper extremity numbness. Her physical examination revealed normal strength and reflexes. She had diminished response to pinprick in both legs to the knees and in both hands to the wrists. Vibration sense was diminished in the bilateral lower extremities.

A glycosylated hemoglobin (HbA1c) level was 6.2%. Methylmalonic acid was 69 nmol per liter (normal range, 45-325). Antibodies to Borrelia burgdorferi and Treponema pallidum were absent. Impaired glucose metabolism was the leading diagnosis for her polyneuropathy, and it was recommended that she undergo an oral glucose tolerance test. Electromyography was not performed.

The neurological symptoms are now chronic, and importantly, the patient has developed sensory deficits on neurological examination, suggesting worsening of the underlying process. While the paresthesia is now limited to a “stocking/glove” distribution consistent with distal sensory polyneuropathy, there should still be a concern for spinal cord pathology given that the HbA1c level of 6.2 would not explain her initial distribution of symptoms. Myelopathy may mimic peripheral nerve disease if, for example, there is involvement of the dorsal columns leading to sensory deficits of vibration and proprioception. Additionally, the transient episode of upper extremity numbness raises the question of sensory nerve root involvement (ie, sensory radiculopathy). Unexplained abdominal pain could possibly represent the involvement of other nerve roots innervating the abdominal wall. The patient’s episode of focal arm numbness recalls the lancinating radicular pain of tabes dorsalis; however, the negative specific treponemal antibody test excludes neurosyphilis.

The differential diagnosis going forward will be strongly conditioned by the localization of the neurological lesion(s). To differentiate between myelopathy, radiculopathy, and peripheral neuropathy, I would perform nerve conduction studies, magnetic resonance imaging (MRI) of the spinal cord, and cerebrospinal fluid analysis.

The patient began taking a multivitamin, and after weeks her paresthesia had resolved. One month later, she developed an intermittent, throbbing left-sided headache and pain behind the left eye that was worsened with ocular movement. She then noted decreased visual acuity in her left eye that progressed the following month. She denied photophobia, flashers, or floaters.

In the emergency department, visual acuity was 20/25 in her right eye; in the left eye she was only able to count fingers. Extraocular movements of both eyes were normal as was her right pupillary reflex. Red desaturation and a relative afferent papillary defect were present in the left eye. Fundoscopic exam demonstrated left optic disc swelling. The remainder of her cranial nerves were normal. She had pronation of the left upper extremity and mild right finger-to-nose dysmetria. Muscle tone, strength, sensation, and deep tendon reflexes were normal.

The improvement in the sensory symptoms was unlikely to be related to the nutritional intervention and provides a clue to an underlying waxing and waning illness. That interpretation is supported by the subsequent development of new visual symptoms and signs, which point to optic nerve pathology. Optic neuropathy has a broad differential diagnosis that includes ischemic, metabolic, toxic, and compressive causes. Eye pain, swelling of the optic disc, and prominent impairment of color vision all point to the more specific syndrome of optic neuritis caused by infections (including both Treponema pallidum and Borrelia species), systemic autoimmune diseases (systemic lupus erythematosus or Sjogren’s syndrome), and central nervous system (CNS) demyelinating diseases. Of these, inflammatory demyelinating processes would be the likeliest explanation of intermittent and improving neurologic findings.

With relapsing symptoms and findings that are separate in distribution and time, two diagnoses become most likely, and both of these are most often diagnosed in young women. MS is common, and optic neuritis occurs in more than 50% of patients over the course of illness. Neuromyelitis optica spectrum disorder (NMOSD) is a rare condition that can exist in isolation or be associated with other autoimmune illnesses. While these entities are difficult to differentiate clinically, neuroimaging that demonstrates extensive intracerebral demyelinating lesions and cerebrospinal fluid with oligoclonal bands favor MS, whereas extensive, predominant spinal cord involvement is suggestive of NMOSD. Approximately 70% of NMO patients harbor an antibody directed against the aquaporin-4 channel, and these antibodies are not seen in patients with MS. A milder NMO-like disorder has also been associated with antimyelin oligodendrocyte antibodies (MOG).

Testing for antinuclear antibodies, anti–double-stranded DNA, anti-Ro (SSA), and anti-La (SSB) antibodies was negative. The level of C3 was 162 mg per deciliter (normal range 81-157) and C4 38 (normal range 13-39). T-spot testing for latent tuberculosis was negative.

There is no serological evidence of active systemic lupus erythematosus or Sjogren’s syndrome. The pretest probability of CNS tuberculosis was low in light of her presenting complaints, relatively protracted course, and overall clinical stability without antituberculous therapy. Tests for latent tuberculosis infection have significant limitations of both sensitivity and specificity for the diagnosis of active disease.

Optical coherence tomography showed optic disc edema in the left eye only. MRI of the head with contrast revealed abnormal signal intensity involving the posterior aspect of the pons, right middle cerebellar peduncle, anterior left temporal lobe, bilateral periventricular white matter, subcortical white matter of the frontal lobes bilaterally, and medulla with abnormal signal and enhancement of the left optic nerve (Figure, Panel A). MRI of the cervical and thoracic spine demonstrated multifocal demyelinating lesions at C3, C4, C7, T4, T5, T7, and T8 (Figure, Panel B). The lesions were not longitudinally extensive. There was no significant postcontrast enhancement to suggest active demyelination.

The cerebrospinal fluid analysis revealed glucose of 105 mg per deciliter and a total protein of 26.1 mg per deciliter. In the fourth tube, there were 20 red cells per cubic and four white cells with a differential of 62% neutrophils, 35% lymphocytes, and 3% monocytes. Epstein-Barr and herpes simplex virus DNA were negative. A Venereal Disease Research Laboratory test was negative. Multiple oligoclonal IgG bands were identified only in the cerebrospinal fluid. Aquaporin-4 IgG and MOG antibodies were negative.

In addition to the expected finding of enhancement of the optic nerve, MRI demonstrated numerous multifocal white matter lesions throughout the cerebrum, brainstem, and spinal cord. Many of the lesions were in “silent” areas, which is not directly attributable to specific symptoms, but several did correlate with the subtler deficits of weakness and dysmetria that were noted on examination. Although such lesions may be seen with a diverse group of systemic diseases including adrenal leukodystrophy, sarcoidosis, Behcet’s, cerebral lupus, and vasculitis, primary CNS inflammatory demyelinating diseases are much more likely. The extensive distribution of demyelination argues against NMOSD. The negative aquaporin-4 and MOG assays support this conclusion. Not all multifocal CNS demyelination is caused by MS and can be seen in posterior reversible encephalopathy syndrome, cerebral autosomal dominant arteriopathy with subcortical infarcts and leukoencephalopathy, and adult polyglucosan body disease. Osmotic demyelination is increasingly being recognized as a process that can be more widespread rather than just being limited to the pons. Viral infections of the CNS such as the JC virus (PML) may also provoke multifocal demyelination. Acute disseminated encephalomyelitis is most often seen during childhood, usually after vaccination or after an infectious prodrome. The tempo of the progression of these other diseases tends to be much more rapid than this woman’s course, and often, the neurological deficits are more profound and debilitating. The clinical presentation of sensory-predominant myelopathy, followed by optic neuritis, absence of systemic inflammatory signs or laboratory markers, exclusion of other relevant diseases, multifocal white matter lesions on imaging, minimal pleocytosis, and presence of oligoclonal bands in cerebrospinal fluid, all point to a diagnosis of relapsing-remitting MS.

The patient was diagnosed with MS. She was admitted to the neurology service and treated with 1,000 mg IV methylprednisolone for 3 days with a prompt improvement in her vision. She was started on natalizumab without a relapse of symptoms over the past year.

Multiple sclerosis is a chronic demyelinating disease of the CNS.¹ The diagnosis of MS has classically been based upon compatible clinical and radiographic evidence of pathology that is disseminated in space and time. Patients typically present with an initial clinically isolated syndrome—involving changes in vision, sensation, strength, mobility, or cognition—for which there is radiographic evidence of demyelination.² A diagnosis of clinically definite MS is then often made based on a subsequent relapse of symptoms.³

An interval from initial symptoms has been central to the diagnosis of MS (“lesions disseminated in time”). However, recent evidence questions this diagnostic paradigm, and a more rapid diagnosis of MS has been recommended. This recommendation is reflected in the updated McDonald criteria, according to which, if a clinical presentation is supported by the presence of oligoclonal bands in the cerebrospinal fluid, a diagnosis can be made on the basis of radiographic evidence of dissemination of disease in space, without evidence of dissemination in time.⁴ The importance of such early diagnosis has been supported by numerous studies that have demonstrated improved clinical outcomes with early therapy.^5-7

Despite the McDonald criteria, delays in definitive diagnosis are common in MS. Patients with MS in Spain were found to experience a 2-year delay from the first onset of symptoms to diagnosis.⁸ In this cohort, patients exhibited delays in presenting to a healthcare provider, as well as delays in diagnosis with an average time from seeing an initial provider to diagnosis of 6 months. When patients who were referred for a demyelinating episode were surveyed, over a third reported a prior suggestive event.⁹ The time from the first suggestive episode to referral to a neurologist for a recognized demyelinating event was 46 months. Other studies have shown that delays in diagnosis are especially common in younger patients, those with primary progressive MS, and those with comorbid disease.^10,11

Misapplication of an MS diagnosis also occurs frequently. In one case series, such misapplication was found most often in cases involving migraine, fibromyalgia, psychogenic disorders, and NMOSD.¹² NMOSD is distinguished from MS by the presence of typical brain and spine findings on MRI.¹³ Antibodies to aquaporin-4 are highly specific and moderately sensitive for the disease.¹⁴ It is important to distinguish NMOSD from MS as certain disease-modifying drugs used for MS might actually exacerbate NMOSD.¹⁵ A lesion that traverses over three or more contiguous vertebral segments with predominant involvement of central gray matter (ie, longitudinally extensive transverse myelitis) on MRI is the most distinct finding of NMOSD. In contrast, similar to our patient, short and often multiple lesions are demonstrated on spinal cord MRI in patients with MS. Sensitive and specific findings of brain MRI in patients with MS include the presence of lateral ventricle and inferior temporal lobe lesion, Dawson’s fingers, central vein sign, or an S-shaped U-fiber lesion. In NMOSD, brain MRI might reveal periependymal lesions surrounding the ventricular system.

This case highlights the diagnostic challenges related to presentations of a waxing and waning neurological process. At the time of the second evaluation, the presentation was interpreted as a length-dependent polyneuropathy due to glucose intolerance. Our patient’s relatively normal HbA1c, subacute onset of neuropathic symptoms (ie, <4 weeks), sensory and motor complaints, and onset in the upper extremities suggested an alternative diagnosis to prediabetes. Once the patient presented with optic neuritis, the cause of the initial symptoms was obvious, but then, hindsight is 20/20.

• Early treatment of MS results in improved clinical outcomes.
• Delays in the definitive diagnosis of MS are common, especially in younger patients, those with primary progressive MS, and those with comorbid disease.
• If a clinical presentation is supported by the presence of oligoclonal bands in the cerebrospinal fluid, a diagnosis of MS can be made on the basis of radiographic evidence of dissemination of disease in space, without evidence of dissemination in time.

Acknowledgments

The authors wish to thank Rabih Geha, MD, and Gurpreet Dhaliwal, MD, for providing feedback on an earlier version of this manuscript.

In surgical comanagement (SCM), surgeons and hospitalists share responsibility of care for surgical patients. While SCM has been increasingly utilized, many of the reported models are a modification of the consultation model, in which a group of rotating hospitalists, internists, or geriatricians care for the surgical patients, often after medical complications may have occured.^1-4

In August 2012, we implemented SCM in Orthopedic and Neurosurgery services at our institution.⁵ This model is unique because the same Internal Medicine hospitalists are dedicated year round to the same surgical service. SCM hospitalists see patients on their assigned surgical service only; they do not see patients on the Internal Medicine service. After the first year of implementing SCM, we conducted a propensity score–weighted study with 17,057 discharges in the pre-SCM group (January 2009 to July 2012) and 5,533 discharges in the post-SCM group (September 2012 to September 2013).⁵ In this study, SCM was associated with a decrease in medical complications, length of stay (LOS), medical consultations, 30-day readmissions, and cost.⁵

Since SCM requires ongoing investment by institutions, we now report a follow-up study to explore if there were continued improvements in patient outcomes with SCM. In this study, we evaluate if there was a decrease in medical complications, LOS, number of medical consultations, rapid response team calls, and code blues and an increase in patient satisfaction with SCM in Orthopedic and Neurosurgery services between 2012 and 2018.

We included 26,380 discharges from Orthopedic and Neurosurgery services between September 1, 2012, and June 30, 2018, at our academic medical center. We excluded patients discharged in August 2012 as we transitioned to the SCM model. Our Institutional Review Board exempted this study from further review.

SCM Structure

SCM structure was detailed in a prior article.⁵ We have 3.0 clinical full-time equivalents on the Orthopedic surgery SCM service and 1.2 on the Neurosurgery SCM service. On weekdays, during the day (8 am to 5 pm), there are two SCM hospitalists on Orthopedic surgery service and one on Neurosurgery service. One SCM hospitalist is on call every week and takes after-hours calls from both surgical services and sees patients on both services on the weekend.

During the day, SCM hospitalists receive the first call for medical issues. After 5 pm and on weekends and holidays, surgical services take all calls first and reach out to the on-call SCM hospitalist for any medical issues for which they need assistance. Surgery service is the primary team and does the discharge summaries. SCM hospitalists write any medical orders as needed. Medical students, physician assistant students, medicine housestaff, and geriatric medicine fellows rotate through SCM. SCM hospitalists directly communicate with the surgical service and not through the learners. There are no advanced practice providers on SCM service. Surgery housestaff attend the multidisciplinary team care rounds with the case manager, social worker, rehabilitation services, and pharmacy with ad hoc presence of SCM hospitalists for selected patients. SCM hospitalists often see sick patients with the surgery service at the bedside, and they work together with the surgery service on order sets, quality improvement projects, and scholarly work.

SCM hospitalists screen the entire patient list on their assigned surgery service each day. After screening the patient list, SCM hospitalists formally see select patients with preventable or active medical conditions and write notes on the patient’s chart. There are no set criteria to determine which patients would be seen by SCM. This is because surgeries can decompensate stable medical conditions or new unexpected medical complications may occur. Additionally, in our prior study, we reported that SCM reduced medical complications and LOS regardless of age or patient acuity.⁵

Outcomes

Our primary outcome was proportion of patients with ≥1 medical complication (sepsis, pneumonia, urinary tract infection, delirium, acute kidney injury, atrial fibrillation, or ileus). Our secondary outcomes included mean LOS, proportion of patients with ≥2 medical consultations, rapid response team calls, code blues, and top-box patient satisfaction score. Though cost is an important consideration in implementing SCM, limited financial data were available. However, since LOS is a key component in calculating direct costs,⁶ we estimated the cost savings per discharge using mean direct cost per day and the difference in mean LOS between pre- and post-SCM groups.⁵

We defined medical complications using International Classification of Disease (ICD) Codes 9 or 10 that were coded as “not present on admission” (Appendix 1). We used Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey for three questions for patient satisfaction: Did doctors treat with courtesy and respect, listen carefully, and explain things in a way you could understand?

Statistical Analysis

We used regression analysis to assess trends in patient characteristics by year (Appendix 2). Logistic regression with logit link was used to assess the yearly change in our binary outcomes (proportion of patients with ≥1 medical complication, those with ≥2 medical consultations, rapid response team calls, code blue, and top-box patient satisfaction score) and reported odds ratios. Gamma regression with identity link was performed for our continuous outcome (LOS). Beta coefficient was reported to estimate the yearly change in LOS under their original scales. Age, primary insurance, race, Charlson comorbidity score, general or regional anesthesia, surgical service, and duration of surgery were adjusted in the regression analyses for outcomes. SAS 9.4 was used for analysis.

Patient characteristics are shown in Table 1. Overall, 62.8% patients were discharged from Orthopedic surgery service, 72.5% patients underwent elective surgery, and 88.8% received general anesthesia. Between 2012 and 2018, there was a significant increase in the median age of patients (from 60 years to 63 years), mean Charlson comorbidity score increased from 1.07 to 1.46, and median case mix index, a measure of patient acuity, increased from 2.10 to 2.36 (Appendix 2).

Comparing pre-SCM unadjusted rates reported in our prior study (January 2009 to July 2012) to post-SCM (September 2012 to June 2018; Appendix 3), patients with ≥1 medical complication decreased from 10.1% to 6.1%, LOS (mean ± standard deviation) changed from 5.4 ± 2.2 days to 4.6 ± 5.8 days, patients with ≥2 medical consultations decreased from 19.4% to 9.2%, rapid response team calls changed from 1% to 0.9%, code blues changed from 0.3% to 0.2%, and patients with top-box patient satisfaction score increased from 86.4% to 94.2%.⁵

In the adjusted analysis from 2012 to 2018, the odds of patients with ≥1 medical complication decreased by 3.8% per year (P = .01), estimated LOS decreased by 0.3 days per year (P < .0001), and the odds of rapid response team calls decreased by 12.2% per year (P = .001; Table 2). Changes over time in the odds of patients with ≥2 medical consultations, code blues, or top-box patient satisfaction score were not statistically significant (Table 2). Based on the LOS reduction pre- to post-SCM, there were estimated average direct cost savings of $3,424 per discharge between 2012 and 2018.

Since the implementation of SCM on Orthopedic and Neurosurgery services at our institution, there was a decrease in medical complications, LOS, and rapid response team calls. To our knowledge, this is one of the largest studies evaluating the benefits of SCM over 5.8 years. Similar to our prior studies on this SCM model of care,^5,7 other studies have reported a decrease in medical complications,^8-10 LOS,^11-13 and cost of care¹⁴ with SCM.

While the changes in the unadjusted rates of outcomes over the years appeared to be small, while our patient population became older and sicker, there were significant changes in several of our outcomes in the adjusted analysis. We believe that SCM hospitalists have developed a skill set and understanding of these surgical patients over time and can manage more medically complex patients without an increase in medical complications or LOS. We attribute this to our unique SCM model in which the same hospitalists stay year round on the same surgical service. SCM hospitalists have built trusting relationships with the surgical team with greater involvement in decision making, care planning, and patient selection. With minimal turnover in the SCM group and with ongoing learning, SCM hospitalists can anticipate fluid or pain medication requirements after specific surgeries and the surgery-specific medical complications. SCM hospitalists are available on the patient units to provide timely intervention in case of medical deterioration; answer any questions from patients, families, or nursing while the surgical teams may be in the operating room; and coordinate with other medical consultants or outpatient providers as needed.

This study has several limitations. This is a single-center study at an academic institution, limited to two surgical services. We did not have a control group and multiple hospital-­wide interventions may have affected these outcomes. This is an observational study in which unobserved variables may bias the results. We used ICD codes to identify medical complications, which relies on the quality of physician documentation. While our response rate of 21.1% for HCAHPS was comparable to the national average of 26.7%, it may not reliably represent our patient population.¹⁵ Lastly, we had limited financial data.

With the move toward value-based payment and increasing medical complexity of surgical patients, SCM by hospitalists may deliver high-quality care.

In surgical comanagement (SCM), surgeons and hospitalists share responsibility of care for surgical patients. While SCM has been increasingly utilized, many of the reported models are a modification of the consultation model, in which a group of rotating hospitalists, internists, or geriatricians care for the surgical patients, often after medical complications may have occured.^1-4

In August 2012, we implemented SCM in Orthopedic and Neurosurgery services at our institution.⁵ This model is unique because the same Internal Medicine hospitalists are dedicated year round to the same surgical service. SCM hospitalists see patients on their assigned surgical service only; they do not see patients on the Internal Medicine service. After the first year of implementing SCM, we conducted a propensity score–weighted study with 17,057 discharges in the pre-SCM group (January 2009 to July 2012) and 5,533 discharges in the post-SCM group (September 2012 to September 2013).⁵ In this study, SCM was associated with a decrease in medical complications, length of stay (LOS), medical consultations, 30-day readmissions, and cost.⁵

Since SCM requires ongoing investment by institutions, we now report a follow-up study to explore if there were continued improvements in patient outcomes with SCM. In this study, we evaluate if there was a decrease in medical complications, LOS, number of medical consultations, rapid response team calls, and code blues and an increase in patient satisfaction with SCM in Orthopedic and Neurosurgery services between 2012 and 2018.

We included 26,380 discharges from Orthopedic and Neurosurgery services between September 1, 2012, and June 30, 2018, at our academic medical center. We excluded patients discharged in August 2012 as we transitioned to the SCM model. Our Institutional Review Board exempted this study from further review.

SCM Structure

SCM structure was detailed in a prior article.⁵ We have 3.0 clinical full-time equivalents on the Orthopedic surgery SCM service and 1.2 on the Neurosurgery SCM service. On weekdays, during the day (8 am to 5 pm), there are two SCM hospitalists on Orthopedic surgery service and one on Neurosurgery service. One SCM hospitalist is on call every week and takes after-hours calls from both surgical services and sees patients on both services on the weekend.

During the day, SCM hospitalists receive the first call for medical issues. After 5 pm and on weekends and holidays, surgical services take all calls first and reach out to the on-call SCM hospitalist for any medical issues for which they need assistance. Surgery service is the primary team and does the discharge summaries. SCM hospitalists write any medical orders as needed. Medical students, physician assistant students, medicine housestaff, and geriatric medicine fellows rotate through SCM. SCM hospitalists directly communicate with the surgical service and not through the learners. There are no advanced practice providers on SCM service. Surgery housestaff attend the multidisciplinary team care rounds with the case manager, social worker, rehabilitation services, and pharmacy with ad hoc presence of SCM hospitalists for selected patients. SCM hospitalists often see sick patients with the surgery service at the bedside, and they work together with the surgery service on order sets, quality improvement projects, and scholarly work.

SCM hospitalists screen the entire patient list on their assigned surgery service each day. After screening the patient list, SCM hospitalists formally see select patients with preventable or active medical conditions and write notes on the patient’s chart. There are no set criteria to determine which patients would be seen by SCM. This is because surgeries can decompensate stable medical conditions or new unexpected medical complications may occur. Additionally, in our prior study, we reported that SCM reduced medical complications and LOS regardless of age or patient acuity.⁵

Outcomes

Our primary outcome was proportion of patients with ≥1 medical complication (sepsis, pneumonia, urinary tract infection, delirium, acute kidney injury, atrial fibrillation, or ileus). Our secondary outcomes included mean LOS, proportion of patients with ≥2 medical consultations, rapid response team calls, code blues, and top-box patient satisfaction score. Though cost is an important consideration in implementing SCM, limited financial data were available. However, since LOS is a key component in calculating direct costs,⁶ we estimated the cost savings per discharge using mean direct cost per day and the difference in mean LOS between pre- and post-SCM groups.⁵

We defined medical complications using International Classification of Disease (ICD) Codes 9 or 10 that were coded as “not present on admission” (Appendix 1). We used Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey for three questions for patient satisfaction: Did doctors treat with courtesy and respect, listen carefully, and explain things in a way you could understand?

Statistical Analysis

We used regression analysis to assess trends in patient characteristics by year (Appendix 2). Logistic regression with logit link was used to assess the yearly change in our binary outcomes (proportion of patients with ≥1 medical complication, those with ≥2 medical consultations, rapid response team calls, code blue, and top-box patient satisfaction score) and reported odds ratios. Gamma regression with identity link was performed for our continuous outcome (LOS). Beta coefficient was reported to estimate the yearly change in LOS under their original scales. Age, primary insurance, race, Charlson comorbidity score, general or regional anesthesia, surgical service, and duration of surgery were adjusted in the regression analyses for outcomes. SAS 9.4 was used for analysis.

Patient characteristics are shown in Table 1. Overall, 62.8% patients were discharged from Orthopedic surgery service, 72.5% patients underwent elective surgery, and 88.8% received general anesthesia. Between 2012 and 2018, there was a significant increase in the median age of patients (from 60 years to 63 years), mean Charlson comorbidity score increased from 1.07 to 1.46, and median case mix index, a measure of patient acuity, increased from 2.10 to 2.36 (Appendix 2).

Comparing pre-SCM unadjusted rates reported in our prior study (January 2009 to July 2012) to post-SCM (September 2012 to June 2018; Appendix 3), patients with ≥1 medical complication decreased from 10.1% to 6.1%, LOS (mean ± standard deviation) changed from 5.4 ± 2.2 days to 4.6 ± 5.8 days, patients with ≥2 medical consultations decreased from 19.4% to 9.2%, rapid response team calls changed from 1% to 0.9%, code blues changed from 0.3% to 0.2%, and patients with top-box patient satisfaction score increased from 86.4% to 94.2%.⁵

In the adjusted analysis from 2012 to 2018, the odds of patients with ≥1 medical complication decreased by 3.8% per year (P = .01), estimated LOS decreased by 0.3 days per year (P < .0001), and the odds of rapid response team calls decreased by 12.2% per year (P = .001; Table 2). Changes over time in the odds of patients with ≥2 medical consultations, code blues, or top-box patient satisfaction score were not statistically significant (Table 2). Based on the LOS reduction pre- to post-SCM, there were estimated average direct cost savings of $3,424 per discharge between 2012 and 2018.

Since the implementation of SCM on Orthopedic and Neurosurgery services at our institution, there was a decrease in medical complications, LOS, and rapid response team calls. To our knowledge, this is one of the largest studies evaluating the benefits of SCM over 5.8 years. Similar to our prior studies on this SCM model of care,^5,7 other studies have reported a decrease in medical complications,^8-10 LOS,^11-13 and cost of care¹⁴ with SCM.

While the changes in the unadjusted rates of outcomes over the years appeared to be small, while our patient population became older and sicker, there were significant changes in several of our outcomes in the adjusted analysis. We believe that SCM hospitalists have developed a skill set and understanding of these surgical patients over time and can manage more medically complex patients without an increase in medical complications or LOS. We attribute this to our unique SCM model in which the same hospitalists stay year round on the same surgical service. SCM hospitalists have built trusting relationships with the surgical team with greater involvement in decision making, care planning, and patient selection. With minimal turnover in the SCM group and with ongoing learning, SCM hospitalists can anticipate fluid or pain medication requirements after specific surgeries and the surgery-specific medical complications. SCM hospitalists are available on the patient units to provide timely intervention in case of medical deterioration; answer any questions from patients, families, or nursing while the surgical teams may be in the operating room; and coordinate with other medical consultants or outpatient providers as needed.

This study has several limitations. This is a single-center study at an academic institution, limited to two surgical services. We did not have a control group and multiple hospital-­wide interventions may have affected these outcomes. This is an observational study in which unobserved variables may bias the results. We used ICD codes to identify medical complications, which relies on the quality of physician documentation. While our response rate of 21.1% for HCAHPS was comparable to the national average of 26.7%, it may not reliably represent our patient population.¹⁵ Lastly, we had limited financial data.

With the move toward value-based payment and increasing medical complexity of surgical patients, SCM by hospitalists may deliver high-quality care.

In surgical comanagement (SCM), surgeons and hospitalists share responsibility of care for surgical patients. While SCM has been increasingly utilized, many of the reported models are a modification of the consultation model, in which a group of rotating hospitalists, internists, or geriatricians care for the surgical patients, often after medical complications may have occured.^1-4

In August 2012, we implemented SCM in Orthopedic and Neurosurgery services at our institution.⁵ This model is unique because the same Internal Medicine hospitalists are dedicated year round to the same surgical service. SCM hospitalists see patients on their assigned surgical service only; they do not see patients on the Internal Medicine service. After the first year of implementing SCM, we conducted a propensity score–weighted study with 17,057 discharges in the pre-SCM group (January 2009 to July 2012) and 5,533 discharges in the post-SCM group (September 2012 to September 2013).⁵ In this study, SCM was associated with a decrease in medical complications, length of stay (LOS), medical consultations, 30-day readmissions, and cost.⁵

Since SCM requires ongoing investment by institutions, we now report a follow-up study to explore if there were continued improvements in patient outcomes with SCM. In this study, we evaluate if there was a decrease in medical complications, LOS, number of medical consultations, rapid response team calls, and code blues and an increase in patient satisfaction with SCM in Orthopedic and Neurosurgery services between 2012 and 2018.

We included 26,380 discharges from Orthopedic and Neurosurgery services between September 1, 2012, and June 30, 2018, at our academic medical center. We excluded patients discharged in August 2012 as we transitioned to the SCM model. Our Institutional Review Board exempted this study from further review.

SCM Structure

SCM structure was detailed in a prior article.⁵ We have 3.0 clinical full-time equivalents on the Orthopedic surgery SCM service and 1.2 on the Neurosurgery SCM service. On weekdays, during the day (8 am to 5 pm), there are two SCM hospitalists on Orthopedic surgery service and one on Neurosurgery service. One SCM hospitalist is on call every week and takes after-hours calls from both surgical services and sees patients on both services on the weekend.

During the day, SCM hospitalists receive the first call for medical issues. After 5 pm and on weekends and holidays, surgical services take all calls first and reach out to the on-call SCM hospitalist for any medical issues for which they need assistance. Surgery service is the primary team and does the discharge summaries. SCM hospitalists write any medical orders as needed. Medical students, physician assistant students, medicine housestaff, and geriatric medicine fellows rotate through SCM. SCM hospitalists directly communicate with the surgical service and not through the learners. There are no advanced practice providers on SCM service. Surgery housestaff attend the multidisciplinary team care rounds with the case manager, social worker, rehabilitation services, and pharmacy with ad hoc presence of SCM hospitalists for selected patients. SCM hospitalists often see sick patients with the surgery service at the bedside, and they work together with the surgery service on order sets, quality improvement projects, and scholarly work.

SCM hospitalists screen the entire patient list on their assigned surgery service each day. After screening the patient list, SCM hospitalists formally see select patients with preventable or active medical conditions and write notes on the patient’s chart. There are no set criteria to determine which patients would be seen by SCM. This is because surgeries can decompensate stable medical conditions or new unexpected medical complications may occur. Additionally, in our prior study, we reported that SCM reduced medical complications and LOS regardless of age or patient acuity.⁵

Outcomes

Our primary outcome was proportion of patients with ≥1 medical complication (sepsis, pneumonia, urinary tract infection, delirium, acute kidney injury, atrial fibrillation, or ileus). Our secondary outcomes included mean LOS, proportion of patients with ≥2 medical consultations, rapid response team calls, code blues, and top-box patient satisfaction score. Though cost is an important consideration in implementing SCM, limited financial data were available. However, since LOS is a key component in calculating direct costs,⁶ we estimated the cost savings per discharge using mean direct cost per day and the difference in mean LOS between pre- and post-SCM groups.⁵

We defined medical complications using International Classification of Disease (ICD) Codes 9 or 10 that were coded as “not present on admission” (Appendix 1). We used Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey for three questions for patient satisfaction: Did doctors treat with courtesy and respect, listen carefully, and explain things in a way you could understand?

Statistical Analysis

We used regression analysis to assess trends in patient characteristics by year (Appendix 2). Logistic regression with logit link was used to assess the yearly change in our binary outcomes (proportion of patients with ≥1 medical complication, those with ≥2 medical consultations, rapid response team calls, code blue, and top-box patient satisfaction score) and reported odds ratios. Gamma regression with identity link was performed for our continuous outcome (LOS). Beta coefficient was reported to estimate the yearly change in LOS under their original scales. Age, primary insurance, race, Charlson comorbidity score, general or regional anesthesia, surgical service, and duration of surgery were adjusted in the regression analyses for outcomes. SAS 9.4 was used for analysis.

Patient characteristics are shown in Table 1. Overall, 62.8% patients were discharged from Orthopedic surgery service, 72.5% patients underwent elective surgery, and 88.8% received general anesthesia. Between 2012 and 2018, there was a significant increase in the median age of patients (from 60 years to 63 years), mean Charlson comorbidity score increased from 1.07 to 1.46, and median case mix index, a measure of patient acuity, increased from 2.10 to 2.36 (Appendix 2).

Comparing pre-SCM unadjusted rates reported in our prior study (January 2009 to July 2012) to post-SCM (September 2012 to June 2018; Appendix 3), patients with ≥1 medical complication decreased from 10.1% to 6.1%, LOS (mean ± standard deviation) changed from 5.4 ± 2.2 days to 4.6 ± 5.8 days, patients with ≥2 medical consultations decreased from 19.4% to 9.2%, rapid response team calls changed from 1% to 0.9%, code blues changed from 0.3% to 0.2%, and patients with top-box patient satisfaction score increased from 86.4% to 94.2%.⁵

In the adjusted analysis from 2012 to 2018, the odds of patients with ≥1 medical complication decreased by 3.8% per year (P = .01), estimated LOS decreased by 0.3 days per year (P < .0001), and the odds of rapid response team calls decreased by 12.2% per year (P = .001; Table 2). Changes over time in the odds of patients with ≥2 medical consultations, code blues, or top-box patient satisfaction score were not statistically significant (Table 2). Based on the LOS reduction pre- to post-SCM, there were estimated average direct cost savings of $3,424 per discharge between 2012 and 2018.

Since the implementation of SCM on Orthopedic and Neurosurgery services at our institution, there was a decrease in medical complications, LOS, and rapid response team calls. To our knowledge, this is one of the largest studies evaluating the benefits of SCM over 5.8 years. Similar to our prior studies on this SCM model of care,^5,7 other studies have reported a decrease in medical complications,^8-10 LOS,^11-13 and cost of care¹⁴ with SCM.

While the changes in the unadjusted rates of outcomes over the years appeared to be small, while our patient population became older and sicker, there were significant changes in several of our outcomes in the adjusted analysis. We believe that SCM hospitalists have developed a skill set and understanding of these surgical patients over time and can manage more medically complex patients without an increase in medical complications or LOS. We attribute this to our unique SCM model in which the same hospitalists stay year round on the same surgical service. SCM hospitalists have built trusting relationships with the surgical team with greater involvement in decision making, care planning, and patient selection. With minimal turnover in the SCM group and with ongoing learning, SCM hospitalists can anticipate fluid or pain medication requirements after specific surgeries and the surgery-specific medical complications. SCM hospitalists are available on the patient units to provide timely intervention in case of medical deterioration; answer any questions from patients, families, or nursing while the surgical teams may be in the operating room; and coordinate with other medical consultants or outpatient providers as needed.

This study has several limitations. This is a single-center study at an academic institution, limited to two surgical services. We did not have a control group and multiple hospital-­wide interventions may have affected these outcomes. This is an observational study in which unobserved variables may bias the results. We used ICD codes to identify medical complications, which relies on the quality of physician documentation. While our response rate of 21.1% for HCAHPS was comparable to the national average of 26.7%, it may not reliably represent our patient population.¹⁵ Lastly, we had limited financial data.

With the move toward value-based payment and increasing medical complexity of surgical patients, SCM by hospitalists may deliver high-quality care.

Inpatient consultation is an extremely common practice with the potential to improve patient outcomes significantly.^1-3 However, variability in consultation practices may be risky for patients. In addition to underuse when the benefit is clear, the overuse of consultation may lead to additional testing and therapies, increased length of stay (LOS) and costs, conflicting recommendations, and opportunities for communication breakdown.

Consultation use is often at the discretion of individual providers. While this decision is frequently driven by patient needs, significant variation in consultation practices not fully explained by patient factors exists.¹ Prior work has described hospital-level variation¹ and that primary care physicians use more consultation than hospitalists.⁴ However, other factors affecting consultation remain unknown. We sought to explore physician-, patient-, and admission-level factors associated with consultation use on inpatient general medicine services.

Study Design

We conducted a retrospective analysis of data from the University of Chicago Hospitalist Project (UCHP). UCHP is a longstanding study of the care of hospitalized patients admitted to the University of Chicago general medicine services, involving both patient data collection and physician experience surveys.⁵ Data were obtained for enrolled UCHP patients between 2011-2016 from the Center for Research Informatics (CRI). The University of Chicago Institutional Review Board approved this study.

Data Collection

Attendings and patients consented to UCHP participation. Data collection details are described elsewhere.^5,6 Data from EpicCare (EpicSystems Corp, Wisconsin) and Centricity Billing (GE Healthcare, Illinois) were obtained via CRI for all encounters of enrolled UCHP patients during the study period (N = 218,591).

Attending Attribution

We determined attending attribution for admissions as follows: the attending author of the first history and physical (H&P) was assigned. If this was unavailable, the attending author of the first progress note (PN) was assigned. For patients admitted by hospitalists on admitting shifts to nonteaching services (ie, service without residents/students), the author of the first PN was assigned if different from H&P. Where available, attribution was corroborated with call schedules.

Sample and Variables

All encounters containing inpatient admissions to the University of Chicago from May 10, 2011 (Electronic Health Record activation date), through December 31, 2016, were considered for inclusion (N = 51,171, Appendix 1). Admissions including only documentation from ancillary services were excluded (eg, encounters for hemodialysis or physical therapy). Admissions were limited to a length of stay (LOS) ≤ 5 days, corresponding to the average US inpatient LOS of 4.6 days,⁷ to minimize the likelihood of attending handoffs (N = 31,592). If attending attribution was not possible via the above-described methods, the admission was eliminated (N = 3,103; 10.9% of admissions with LOS ≤ 5 days). Finally, the sample was restricted to general medicine service admissions under attendings enrolled in UCHP who completed surveys. After the application of all criteria, 6,153 admissions remained for analysis.

The outcome variable was the number of consultations per admission, determined by counting the unique number of services creating clinical documentation, and subtracting one for the primary team. If the Medical/Surgical intensive care unit (ICU) was a service, then two were subtracted to account for the ICU transfer.

Attending years in practice (ie, years since medical school graduation) and gender were determined from public resources. Practice characteristics were determined from UCHP attending surveys, which address perceptions of workload and satisfaction (Appendix 2).

Patient characteristics (gender, age, Elixhauser Indices) and admission characteristics (LOS, season of admission, payor) were determined from UCHP and CRI data. The Elixhauser Index uses a well-validated system combining the presence/absence of 31 comorbidities to predict mortality and 30-day readmission.⁸ Elixhauser Indices were calculated using the “Creation of Elixhauser Comorbidity Index Scores 1.0” software.⁹ For admissions under hospitalist attendings, teaching/nonteaching team was ascertained via internal teaching service calendars.

Analysis

We used descriptive statistics to examine demographic characteristics. The difference between the lowest and highest quartile consultation use was determined via a two-sample t test. Given the multilevel nature of our count data, we used a mixed-effects Poisson model accounting for within-group variation by clustering on attending and patient (3-level random-effects model). The analysis was done using Stata 15 (StataCorp, Texas).

From 2011 to 2016, 14,848 patients and 88 attendings were enrolled in UCHP; 4,772 patients (32%) and 69 attendings (59.4%) had data available and were included. Mean LOS was 3.0 days (SD = 1.3). Table 1 describes the characteristics of attendings, patients, and admissions.

Seventy-six percent of admissions included at least one consultation. Consultation use varied widely, ranging from 0 to 10 per admission (mean = 1.39, median = 1; standard deviation [SD] = 1.17). The number of consultations per admission in the highest quartile of consultation frequency (mean = 3.47, median = 3) was 5.7-fold that of the lowest quartile (mean = 0.613, median = 1; P <.001).

In multivariable regression, physician-, patient-, and admission-level characteristics were associated with the differential use of consultation (Table 2). On teaching services, consultations called by hospitalist vs nonhospitalist generalists did not differ (P =.361). However, hospitalists on nonteaching services called 8.6% more consultations than hospitalists on teaching services (P =.02). Attending agreement with survey item “The interruption of my personal life by work is a problem” was associated with 8.2% fewer consultations per admission (P =.002).

Patients older than 75 years received 19% fewer consultations compared with patients younger than 49 years (P <.001). Compared with Medicare, Medicaid admissions had 12.2% fewer consultations (P <.001), whereas privately insured admissions had 10.7% more (P =.001). The number of consultations per admission decreased every year, with 45.3% fewer consultations in 2015 than 2011 (P <.001). Consultations increased by each 22% per day increase in LOS (P <.001).

Our analysis described several physician-, patient-, and admission-level characteristics associated with the use of inpatient consultation. Our results strengthen prior work demonstrating that patient-level factors alone are insufficient to explain consultation variability.¹

Hospitalists on nonteaching services called more consultations, which may reflect a higher workload on these services. Busy hospitalists on nonteaching teams may lack time to delve deeply into clinical problems and require more consultations, especially for work with heavy cognitive loads such as diagnosis. “Outsourcing” tasks when workload increases occurs in other cognitive activities such as teaching.¹⁰ The association between work interrupting personal life and fewer consultations may also implicate the effects of time. Attendings who are experiencing work encroaching on their personal lives may be those spending more time with patients and consulting less. This finding merits further study, especially with increasing concern about balancing time spent in meaningful patient care activities with risk of physician burnout.

This finding could also indicate that trainee participation modifies consultation use for hospitalists. Teaching service teams with more individual members may allow a greater pool of collective knowledge, decreasing the need for consultation to answer clinical questions.¹¹ Interestingly, there was no difference in consultation use between generalists or subspecialists and hospitalists on teaching services, possibly suggesting a unique effect in hospitalists who vary clinical practice depending on team structure. These differences deserve further investigation, with implications for education and resource utilization.

We were surprised by the finding that consultations decreased each year, despite increasing patient complexity and availability of consultation services. This could be explained by a growing emphasis on shortening LOS in our institution, thus shifting consultative care to outpatient settings. Understanding these effects is critically important with growing evidence that consultation improves patient outcomes because these external pressures could lead to unintended consequences for quality or access to care.

Several findings related to patient factors additionally emerged, including age and insurance status. Although related to medical complexity, these effects persist despite adjustment, which raises the question of whether they contribute to the decision to seek consultation. Older patients received fewer consultations, which could reflect the use of more conservative practice models in the elderly,¹² or ageism, which is associated with undertreatment.¹³ With respect to insurance status, Medicaid patients were associated with fewer consultations. This finding is consistent with previous work showing the decreased intensity of hospital services used for Medicaid patients.¹⁴Our study has limitations. Our data were from one large urban academic center that limits generalizability. Although systematic and redundant, attending attribution may have been flawed: incomplete or erroneous documentation could have led to attribution error, and we cannot rule out the possibility of service handoffs. We used a LOS ≤ 5 days to minimize this possibility, but this limits the applicability of our findings to longer admissions. Unsurprisingly, longer LOS correlated with the increased use of consultation even within our restricted sample, and future work should examine the effects of prolonged LOS. As a retrospective analysis, unmeasured confounders due to our limited adjustment will likely explain some findings, although we took steps to address this in our statistical design. Finally, we could not measure patient outcomes and, therefore, cannot determine the value of more or fewer consultations for specific patients or illnesses. Positive and negative outcomes of increased consultation are described, and understanding the impact of consultation is critical for further study.^2,3

We found that the use of consultation on general medicine services varies widely between admissions, with large differences between the highest and lowest frequencies of use. This variation can be partially explained by several physician-, patient-, and admission-level characteristics. Our work may help identify patient and attending groups at high risk for under- or overuse of consultation and guide the subsequent development of interventions to improve value in consultation. One additional consultation over the average LOS of 4.6 days adds $420 per admission or $4.8 billion to the 11.5 million annual Medicare admissions.¹⁵ Increasing research, guidelines, and education on the judicious use of inpatient consultation will be key in maximizing high-value care and improving patient outcomes.

Acknowledgments

The authors would like to acknowledge the invaluable support and assistance of the University of Chicago Hospitalist Project, the Pritzker School of Medicine Summer Research Program, the University of Chicago Center for Quality, and the University of Chicago Center for Health and the Social Sciences (CHeSS). The authors would additionally like to thank John Cursio, PhD, for his support and guidance in statistical analysis for this project.

Disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. The funders had no role in the design of the study; the collection, analysis, and interpretation of the data; or the decision to approve publication of the finished manuscript. Preliminary results of this analysis were presented at the 2018 Society of Hospital Medicine Annual Meeting in Orlando, Florida. All coauthors have seen and agree with the contents of the manuscript. The submission is not under review by any other publication.

Inpatient consultation is an extremely common practice with the potential to improve patient outcomes significantly.^1-3 However, variability in consultation practices may be risky for patients. In addition to underuse when the benefit is clear, the overuse of consultation may lead to additional testing and therapies, increased length of stay (LOS) and costs, conflicting recommendations, and opportunities for communication breakdown.

Consultation use is often at the discretion of individual providers. While this decision is frequently driven by patient needs, significant variation in consultation practices not fully explained by patient factors exists.¹ Prior work has described hospital-level variation¹ and that primary care physicians use more consultation than hospitalists.⁴ However, other factors affecting consultation remain unknown. We sought to explore physician-, patient-, and admission-level factors associated with consultation use on inpatient general medicine services.

Study Design

We conducted a retrospective analysis of data from the University of Chicago Hospitalist Project (UCHP). UCHP is a longstanding study of the care of hospitalized patients admitted to the University of Chicago general medicine services, involving both patient data collection and physician experience surveys.⁵ Data were obtained for enrolled UCHP patients between 2011-2016 from the Center for Research Informatics (CRI). The University of Chicago Institutional Review Board approved this study.

Data Collection

Attendings and patients consented to UCHP participation. Data collection details are described elsewhere.^5,6 Data from EpicCare (EpicSystems Corp, Wisconsin) and Centricity Billing (GE Healthcare, Illinois) were obtained via CRI for all encounters of enrolled UCHP patients during the study period (N = 218,591).

Attending Attribution

We determined attending attribution for admissions as follows: the attending author of the first history and physical (H&P) was assigned. If this was unavailable, the attending author of the first progress note (PN) was assigned. For patients admitted by hospitalists on admitting shifts to nonteaching services (ie, service without residents/students), the author of the first PN was assigned if different from H&P. Where available, attribution was corroborated with call schedules.

Sample and Variables

All encounters containing inpatient admissions to the University of Chicago from May 10, 2011 (Electronic Health Record activation date), through December 31, 2016, were considered for inclusion (N = 51,171, Appendix 1). Admissions including only documentation from ancillary services were excluded (eg, encounters for hemodialysis or physical therapy). Admissions were limited to a length of stay (LOS) ≤ 5 days, corresponding to the average US inpatient LOS of 4.6 days,⁷ to minimize the likelihood of attending handoffs (N = 31,592). If attending attribution was not possible via the above-described methods, the admission was eliminated (N = 3,103; 10.9% of admissions with LOS ≤ 5 days). Finally, the sample was restricted to general medicine service admissions under attendings enrolled in UCHP who completed surveys. After the application of all criteria, 6,153 admissions remained for analysis.

The outcome variable was the number of consultations per admission, determined by counting the unique number of services creating clinical documentation, and subtracting one for the primary team. If the Medical/Surgical intensive care unit (ICU) was a service, then two were subtracted to account for the ICU transfer.

Attending years in practice (ie, years since medical school graduation) and gender were determined from public resources. Practice characteristics were determined from UCHP attending surveys, which address perceptions of workload and satisfaction (Appendix 2).

Patient characteristics (gender, age, Elixhauser Indices) and admission characteristics (LOS, season of admission, payor) were determined from UCHP and CRI data. The Elixhauser Index uses a well-validated system combining the presence/absence of 31 comorbidities to predict mortality and 30-day readmission.⁸ Elixhauser Indices were calculated using the “Creation of Elixhauser Comorbidity Index Scores 1.0” software.⁹ For admissions under hospitalist attendings, teaching/nonteaching team was ascertained via internal teaching service calendars.

Analysis

We used descriptive statistics to examine demographic characteristics. The difference between the lowest and highest quartile consultation use was determined via a two-sample t test. Given the multilevel nature of our count data, we used a mixed-effects Poisson model accounting for within-group variation by clustering on attending and patient (3-level random-effects model). The analysis was done using Stata 15 (StataCorp, Texas).

From 2011 to 2016, 14,848 patients and 88 attendings were enrolled in UCHP; 4,772 patients (32%) and 69 attendings (59.4%) had data available and were included. Mean LOS was 3.0 days (SD = 1.3). Table 1 describes the characteristics of attendings, patients, and admissions.

Seventy-six percent of admissions included at least one consultation. Consultation use varied widely, ranging from 0 to 10 per admission (mean = 1.39, median = 1; standard deviation [SD] = 1.17). The number of consultations per admission in the highest quartile of consultation frequency (mean = 3.47, median = 3) was 5.7-fold that of the lowest quartile (mean = 0.613, median = 1; P <.001).

In multivariable regression, physician-, patient-, and admission-level characteristics were associated with the differential use of consultation (Table 2). On teaching services, consultations called by hospitalist vs nonhospitalist generalists did not differ (P =.361). However, hospitalists on nonteaching services called 8.6% more consultations than hospitalists on teaching services (P =.02). Attending agreement with survey item “The interruption of my personal life by work is a problem” was associated with 8.2% fewer consultations per admission (P =.002).

Patients older than 75 years received 19% fewer consultations compared with patients younger than 49 years (P <.001). Compared with Medicare, Medicaid admissions had 12.2% fewer consultations (P <.001), whereas privately insured admissions had 10.7% more (P =.001). The number of consultations per admission decreased every year, with 45.3% fewer consultations in 2015 than 2011 (P <.001). Consultations increased by each 22% per day increase in LOS (P <.001).

Our analysis described several physician-, patient-, and admission-level characteristics associated with the use of inpatient consultation. Our results strengthen prior work demonstrating that patient-level factors alone are insufficient to explain consultation variability.¹

Hospitalists on nonteaching services called more consultations, which may reflect a higher workload on these services. Busy hospitalists on nonteaching teams may lack time to delve deeply into clinical problems and require more consultations, especially for work with heavy cognitive loads such as diagnosis. “Outsourcing” tasks when workload increases occurs in other cognitive activities such as teaching.¹⁰ The association between work interrupting personal life and fewer consultations may also implicate the effects of time. Attendings who are experiencing work encroaching on their personal lives may be those spending more time with patients and consulting less. This finding merits further study, especially with increasing concern about balancing time spent in meaningful patient care activities with risk of physician burnout.

This finding could also indicate that trainee participation modifies consultation use for hospitalists. Teaching service teams with more individual members may allow a greater pool of collective knowledge, decreasing the need for consultation to answer clinical questions.¹¹ Interestingly, there was no difference in consultation use between generalists or subspecialists and hospitalists on teaching services, possibly suggesting a unique effect in hospitalists who vary clinical practice depending on team structure. These differences deserve further investigation, with implications for education and resource utilization.

We were surprised by the finding that consultations decreased each year, despite increasing patient complexity and availability of consultation services. This could be explained by a growing emphasis on shortening LOS in our institution, thus shifting consultative care to outpatient settings. Understanding these effects is critically important with growing evidence that consultation improves patient outcomes because these external pressures could lead to unintended consequences for quality or access to care.

Several findings related to patient factors additionally emerged, including age and insurance status. Although related to medical complexity, these effects persist despite adjustment, which raises the question of whether they contribute to the decision to seek consultation. Older patients received fewer consultations, which could reflect the use of more conservative practice models in the elderly,¹² or ageism, which is associated with undertreatment.¹³ With respect to insurance status, Medicaid patients were associated with fewer consultations. This finding is consistent with previous work showing the decreased intensity of hospital services used for Medicaid patients.¹⁴Our study has limitations. Our data were from one large urban academic center that limits generalizability. Although systematic and redundant, attending attribution may have been flawed: incomplete or erroneous documentation could have led to attribution error, and we cannot rule out the possibility of service handoffs. We used a LOS ≤ 5 days to minimize this possibility, but this limits the applicability of our findings to longer admissions. Unsurprisingly, longer LOS correlated with the increased use of consultation even within our restricted sample, and future work should examine the effects of prolonged LOS. As a retrospective analysis, unmeasured confounders due to our limited adjustment will likely explain some findings, although we took steps to address this in our statistical design. Finally, we could not measure patient outcomes and, therefore, cannot determine the value of more or fewer consultations for specific patients or illnesses. Positive and negative outcomes of increased consultation are described, and understanding the impact of consultation is critical for further study.^2,3

We found that the use of consultation on general medicine services varies widely between admissions, with large differences between the highest and lowest frequencies of use. This variation can be partially explained by several physician-, patient-, and admission-level characteristics. Our work may help identify patient and attending groups at high risk for under- or overuse of consultation and guide the subsequent development of interventions to improve value in consultation. One additional consultation over the average LOS of 4.6 days adds $420 per admission or $4.8 billion to the 11.5 million annual Medicare admissions.¹⁵ Increasing research, guidelines, and education on the judicious use of inpatient consultation will be key in maximizing high-value care and improving patient outcomes.

Acknowledgments

The authors would like to acknowledge the invaluable support and assistance of the University of Chicago Hospitalist Project, the Pritzker School of Medicine Summer Research Program, the University of Chicago Center for Quality, and the University of Chicago Center for Health and the Social Sciences (CHeSS). The authors would additionally like to thank John Cursio, PhD, for his support and guidance in statistical analysis for this project.

Disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. The funders had no role in the design of the study; the collection, analysis, and interpretation of the data; or the decision to approve publication of the finished manuscript. Preliminary results of this analysis were presented at the 2018 Society of Hospital Medicine Annual Meeting in Orlando, Florida. All coauthors have seen and agree with the contents of the manuscript. The submission is not under review by any other publication.

Inpatient consultation is an extremely common practice with the potential to improve patient outcomes significantly.^1-3 However, variability in consultation practices may be risky for patients. In addition to underuse when the benefit is clear, the overuse of consultation may lead to additional testing and therapies, increased length of stay (LOS) and costs, conflicting recommendations, and opportunities for communication breakdown.

Consultation use is often at the discretion of individual providers. While this decision is frequently driven by patient needs, significant variation in consultation practices not fully explained by patient factors exists.¹ Prior work has described hospital-level variation¹ and that primary care physicians use more consultation than hospitalists.⁴ However, other factors affecting consultation remain unknown. We sought to explore physician-, patient-, and admission-level factors associated with consultation use on inpatient general medicine services.

Study Design

We conducted a retrospective analysis of data from the University of Chicago Hospitalist Project (UCHP). UCHP is a longstanding study of the care of hospitalized patients admitted to the University of Chicago general medicine services, involving both patient data collection and physician experience surveys.⁵ Data were obtained for enrolled UCHP patients between 2011-2016 from the Center for Research Informatics (CRI). The University of Chicago Institutional Review Board approved this study.

Data Collection

Attendings and patients consented to UCHP participation. Data collection details are described elsewhere.^5,6 Data from EpicCare (EpicSystems Corp, Wisconsin) and Centricity Billing (GE Healthcare, Illinois) were obtained via CRI for all encounters of enrolled UCHP patients during the study period (N = 218,591).

Attending Attribution

We determined attending attribution for admissions as follows: the attending author of the first history and physical (H&P) was assigned. If this was unavailable, the attending author of the first progress note (PN) was assigned. For patients admitted by hospitalists on admitting shifts to nonteaching services (ie, service without residents/students), the author of the first PN was assigned if different from H&P. Where available, attribution was corroborated with call schedules.

Sample and Variables

All encounters containing inpatient admissions to the University of Chicago from May 10, 2011 (Electronic Health Record activation date), through December 31, 2016, were considered for inclusion (N = 51,171, Appendix 1). Admissions including only documentation from ancillary services were excluded (eg, encounters for hemodialysis or physical therapy). Admissions were limited to a length of stay (LOS) ≤ 5 days, corresponding to the average US inpatient LOS of 4.6 days,⁷ to minimize the likelihood of attending handoffs (N = 31,592). If attending attribution was not possible via the above-described methods, the admission was eliminated (N = 3,103; 10.9% of admissions with LOS ≤ 5 days). Finally, the sample was restricted to general medicine service admissions under attendings enrolled in UCHP who completed surveys. After the application of all criteria, 6,153 admissions remained for analysis.

The outcome variable was the number of consultations per admission, determined by counting the unique number of services creating clinical documentation, and subtracting one for the primary team. If the Medical/Surgical intensive care unit (ICU) was a service, then two were subtracted to account for the ICU transfer.

Attending years in practice (ie, years since medical school graduation) and gender were determined from public resources. Practice characteristics were determined from UCHP attending surveys, which address perceptions of workload and satisfaction (Appendix 2).

Patient characteristics (gender, age, Elixhauser Indices) and admission characteristics (LOS, season of admission, payor) were determined from UCHP and CRI data. The Elixhauser Index uses a well-validated system combining the presence/absence of 31 comorbidities to predict mortality and 30-day readmission.⁸ Elixhauser Indices were calculated using the “Creation of Elixhauser Comorbidity Index Scores 1.0” software.⁹ For admissions under hospitalist attendings, teaching/nonteaching team was ascertained via internal teaching service calendars.

Analysis

We used descriptive statistics to examine demographic characteristics. The difference between the lowest and highest quartile consultation use was determined via a two-sample t test. Given the multilevel nature of our count data, we used a mixed-effects Poisson model accounting for within-group variation by clustering on attending and patient (3-level random-effects model). The analysis was done using Stata 15 (StataCorp, Texas).

From 2011 to 2016, 14,848 patients and 88 attendings were enrolled in UCHP; 4,772 patients (32%) and 69 attendings (59.4%) had data available and were included. Mean LOS was 3.0 days (SD = 1.3). Table 1 describes the characteristics of attendings, patients, and admissions.

Seventy-six percent of admissions included at least one consultation. Consultation use varied widely, ranging from 0 to 10 per admission (mean = 1.39, median = 1; standard deviation [SD] = 1.17). The number of consultations per admission in the highest quartile of consultation frequency (mean = 3.47, median = 3) was 5.7-fold that of the lowest quartile (mean = 0.613, median = 1; P <.001).

In multivariable regression, physician-, patient-, and admission-level characteristics were associated with the differential use of consultation (Table 2). On teaching services, consultations called by hospitalist vs nonhospitalist generalists did not differ (P =.361). However, hospitalists on nonteaching services called 8.6% more consultations than hospitalists on teaching services (P =.02). Attending agreement with survey item “The interruption of my personal life by work is a problem” was associated with 8.2% fewer consultations per admission (P =.002).

Patients older than 75 years received 19% fewer consultations compared with patients younger than 49 years (P <.001). Compared with Medicare, Medicaid admissions had 12.2% fewer consultations (P <.001), whereas privately insured admissions had 10.7% more (P =.001). The number of consultations per admission decreased every year, with 45.3% fewer consultations in 2015 than 2011 (P <.001). Consultations increased by each 22% per day increase in LOS (P <.001).

Our analysis described several physician-, patient-, and admission-level characteristics associated with the use of inpatient consultation. Our results strengthen prior work demonstrating that patient-level factors alone are insufficient to explain consultation variability.¹

Hospitalists on nonteaching services called more consultations, which may reflect a higher workload on these services. Busy hospitalists on nonteaching teams may lack time to delve deeply into clinical problems and require more consultations, especially for work with heavy cognitive loads such as diagnosis. “Outsourcing” tasks when workload increases occurs in other cognitive activities such as teaching.¹⁰ The association between work interrupting personal life and fewer consultations may also implicate the effects of time. Attendings who are experiencing work encroaching on their personal lives may be those spending more time with patients and consulting less. This finding merits further study, especially with increasing concern about balancing time spent in meaningful patient care activities with risk of physician burnout.

This finding could also indicate that trainee participation modifies consultation use for hospitalists. Teaching service teams with more individual members may allow a greater pool of collective knowledge, decreasing the need for consultation to answer clinical questions.¹¹ Interestingly, there was no difference in consultation use between generalists or subspecialists and hospitalists on teaching services, possibly suggesting a unique effect in hospitalists who vary clinical practice depending on team structure. These differences deserve further investigation, with implications for education and resource utilization.

We were surprised by the finding that consultations decreased each year, despite increasing patient complexity and availability of consultation services. This could be explained by a growing emphasis on shortening LOS in our institution, thus shifting consultative care to outpatient settings. Understanding these effects is critically important with growing evidence that consultation improves patient outcomes because these external pressures could lead to unintended consequences for quality or access to care.

Several findings related to patient factors additionally emerged, including age and insurance status. Although related to medical complexity, these effects persist despite adjustment, which raises the question of whether they contribute to the decision to seek consultation. Older patients received fewer consultations, which could reflect the use of more conservative practice models in the elderly,¹² or ageism, which is associated with undertreatment.¹³ With respect to insurance status, Medicaid patients were associated with fewer consultations. This finding is consistent with previous work showing the decreased intensity of hospital services used for Medicaid patients.¹⁴Our study has limitations. Our data were from one large urban academic center that limits generalizability. Although systematic and redundant, attending attribution may have been flawed: incomplete or erroneous documentation could have led to attribution error, and we cannot rule out the possibility of service handoffs. We used a LOS ≤ 5 days to minimize this possibility, but this limits the applicability of our findings to longer admissions. Unsurprisingly, longer LOS correlated with the increased use of consultation even within our restricted sample, and future work should examine the effects of prolonged LOS. As a retrospective analysis, unmeasured confounders due to our limited adjustment will likely explain some findings, although we took steps to address this in our statistical design. Finally, we could not measure patient outcomes and, therefore, cannot determine the value of more or fewer consultations for specific patients or illnesses. Positive and negative outcomes of increased consultation are described, and understanding the impact of consultation is critical for further study.^2,3

We found that the use of consultation on general medicine services varies widely between admissions, with large differences between the highest and lowest frequencies of use. This variation can be partially explained by several physician-, patient-, and admission-level characteristics. Our work may help identify patient and attending groups at high risk for under- or overuse of consultation and guide the subsequent development of interventions to improve value in consultation. One additional consultation over the average LOS of 4.6 days adds $420 per admission or $4.8 billion to the 11.5 million annual Medicare admissions.¹⁵ Increasing research, guidelines, and education on the judicious use of inpatient consultation will be key in maximizing high-value care and improving patient outcomes.

Acknowledgments

The authors would like to acknowledge the invaluable support and assistance of the University of Chicago Hospitalist Project, the Pritzker School of Medicine Summer Research Program, the University of Chicago Center for Quality, and the University of Chicago Center for Health and the Social Sciences (CHeSS). The authors would additionally like to thank John Cursio, PhD, for his support and guidance in statistical analysis for this project.

Disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. The funders had no role in the design of the study; the collection, analysis, and interpretation of the data; or the decision to approve publication of the finished manuscript. Preliminary results of this analysis were presented at the 2018 Society of Hospital Medicine Annual Meeting in Orlando, Florida. All coauthors have seen and agree with the contents of the manuscript. The submission is not under review by any other publication.

Many provisions of the Affordable Care Act (ACA) have served to support the hospitalized patient. The expansion of Medicaid and the creation of state and federal insurance exchanges for the individual insurance market both significantly lessened the financial burden of hospital care for millions of Americans. Other aspects have proven more controversial, as many of the ACA’s health policy interventions linked to cost and quality in new ways, implementing untested concepts derived from healthcare services research on a national scale.

The Hospital Readmissions Reduction Program (HRRP) was no exception. Based on early research examining readmissions,¹ the ACA included a mandate for the Centers for Medicare and Medicaid Services (CMS) to establish the HRRP. Beginning in Fiscal Year 2013, the HRRP reduced payments for excessive, 30-day, risk-standardized readmissions covering six conditions and procedures. As the third leading cause of 30-day readmissions, chronic obstructive pulmonary disease (COPD) was included in the list of designated HRRP conditions.

This inclusion of COPD in HRRP was not without controversy; analysis of Medicare data from before the ACA’s implementation demonstrated that only half of all readmissions for acute exacerbations of COPD were respiratory-related and only a third were directly related to COPD.² Unsurprisingly, the high proportion of readmissions due to non-COPD-related causes is considered to be one of the leading factors for the failure of COPD readmission reduction programs to find significant reductions in readmissions.³ In this month’s issue of the Journal of Hospital Medicine, Buhr and colleagues explore differential readmission diagnoses following acute exacerbations of COPD using a validated, national, all-payer database.⁴

Like many analyses of payer datasets, this study has several limitations. First, although a large area of the US was included, the data did not include all US states. Further, as the study used multiple cross-sectional data using pooling techniques, it was not truly a longitudinal study. It was additionally limited to 10 months out of the calendar year, missing December and January, which have a high seasonal prevalence of viral respiratory illness. Finally, due to the nature of the data, COPD diagnoses were identified through administrative data known to be highly unreliable for fully capturing admissions for acute exacerbation of COPD.

Despite these limitations, the analysis by Buhr and colleagues provides additional value. They found an overall readmission rate of 17%, with just under half (7.69%) due to recurrent COPD. Patients with COPD-related readmissions were younger, had a higher proportion with Medicaid as the payer, were more frequently discharged home without services, had a shorter length of stay, and had fewer comorbidities.

Most critically, Buhr and colleagues—with a multipayer database—confirmed what researchers found in uni-payer⁵ and site-specific⁶ datasets: over half of readmissions are due to diagnoses other than COPD or respiratory-related causes. Patients readmitted due to other, unrelated diagnoses had a higher mean Elixhauser Comorbidity Index score along with higher rates of congestive heart failure and renal failure. To the practicing hospitalist, this finding supports what our internal clinical voice tells us: sicker patients are readmitted more often and more frequently with conditions unrelated to their index admission diagnosis.

The reaffirmation of the finding that the majority of readmissions are due to nonrespiratory-related causes suggests that perhaps we have a different problem than physicians and policymakers originally thought when adding COPD to the HRRP. Many COPD patients suffer from a polychronic disease, requiring a more holistic approach rather than a traditional, disease-driven, siloed approach focused solely on improving COPD-related care. It may also be true that for other subpopulations of patients with COPD, additional in-hospital and transition of care interventions are required to address patients’ multimorbidity and social determinants of health.

As physicians on the front lines of the readmitted patient, hospitalists are uniquely situated to see the challenges of populations with increasing disease complexity and disease combinations.⁷ The HRRP policy remains controversial. This is due in large part to recent work suggesting that while the HRRP may have helped reduce readmissions, its implementation may have driven the unintended consequence of increased mortality.⁸ Thus, our profession faces an existential challenge to traditional care delivery models targeting diseases. What has not been well parsed by the hospital industry or policymakers is what to do about it.

Readmission of the multimorbid patient, coupled with the challenges of the HRRP, focuses our attention on the need to transition care delivery to a model that is better suited to our patients’ needs: mass-customized, mass-produced service delivery. As physicians, we know that care delivery must be oriented around patients who have many diseases and unique life circumstances. It is our profession’s greatest challenge to collaborate with researchers and administrators to help do this with scale.

Acknowledgments

The authors thank Mary Akel for her assistance with manuscript submission.

Many provisions of the Affordable Care Act (ACA) have served to support the hospitalized patient. The expansion of Medicaid and the creation of state and federal insurance exchanges for the individual insurance market both significantly lessened the financial burden of hospital care for millions of Americans. Other aspects have proven more controversial, as many of the ACA’s health policy interventions linked to cost and quality in new ways, implementing untested concepts derived from healthcare services research on a national scale.

The Hospital Readmissions Reduction Program (HRRP) was no exception. Based on early research examining readmissions,¹ the ACA included a mandate for the Centers for Medicare and Medicaid Services (CMS) to establish the HRRP. Beginning in Fiscal Year 2013, the HRRP reduced payments for excessive, 30-day, risk-standardized readmissions covering six conditions and procedures. As the third leading cause of 30-day readmissions, chronic obstructive pulmonary disease (COPD) was included in the list of designated HRRP conditions.

This inclusion of COPD in HRRP was not without controversy; analysis of Medicare data from before the ACA’s implementation demonstrated that only half of all readmissions for acute exacerbations of COPD were respiratory-related and only a third were directly related to COPD.² Unsurprisingly, the high proportion of readmissions due to non-COPD-related causes is considered to be one of the leading factors for the failure of COPD readmission reduction programs to find significant reductions in readmissions.³ In this month’s issue of the Journal of Hospital Medicine, Buhr and colleagues explore differential readmission diagnoses following acute exacerbations of COPD using a validated, national, all-payer database.⁴

Like many analyses of payer datasets, this study has several limitations. First, although a large area of the US was included, the data did not include all US states. Further, as the study used multiple cross-sectional data using pooling techniques, it was not truly a longitudinal study. It was additionally limited to 10 months out of the calendar year, missing December and January, which have a high seasonal prevalence of viral respiratory illness. Finally, due to the nature of the data, COPD diagnoses were identified through administrative data known to be highly unreliable for fully capturing admissions for acute exacerbation of COPD.

Despite these limitations, the analysis by Buhr and colleagues provides additional value. They found an overall readmission rate of 17%, with just under half (7.69%) due to recurrent COPD. Patients with COPD-related readmissions were younger, had a higher proportion with Medicaid as the payer, were more frequently discharged home without services, had a shorter length of stay, and had fewer comorbidities.

Most critically, Buhr and colleagues—with a multipayer database—confirmed what researchers found in uni-payer⁵ and site-specific⁶ datasets: over half of readmissions are due to diagnoses other than COPD or respiratory-related causes. Patients readmitted due to other, unrelated diagnoses had a higher mean Elixhauser Comorbidity Index score along with higher rates of congestive heart failure and renal failure. To the practicing hospitalist, this finding supports what our internal clinical voice tells us: sicker patients are readmitted more often and more frequently with conditions unrelated to their index admission diagnosis.

The reaffirmation of the finding that the majority of readmissions are due to nonrespiratory-related causes suggests that perhaps we have a different problem than physicians and policymakers originally thought when adding COPD to the HRRP. Many COPD patients suffer from a polychronic disease, requiring a more holistic approach rather than a traditional, disease-driven, siloed approach focused solely on improving COPD-related care. It may also be true that for other subpopulations of patients with COPD, additional in-hospital and transition of care interventions are required to address patients’ multimorbidity and social determinants of health.

As physicians on the front lines of the readmitted patient, hospitalists are uniquely situated to see the challenges of populations with increasing disease complexity and disease combinations.⁷ The HRRP policy remains controversial. This is due in large part to recent work suggesting that while the HRRP may have helped reduce readmissions, its implementation may have driven the unintended consequence of increased mortality.⁸ Thus, our profession faces an existential challenge to traditional care delivery models targeting diseases. What has not been well parsed by the hospital industry or policymakers is what to do about it.

Readmission of the multimorbid patient, coupled with the challenges of the HRRP, focuses our attention on the need to transition care delivery to a model that is better suited to our patients’ needs: mass-customized, mass-produced service delivery. As physicians, we know that care delivery must be oriented around patients who have many diseases and unique life circumstances. It is our profession’s greatest challenge to collaborate with researchers and administrators to help do this with scale.

Acknowledgments

The authors thank Mary Akel for her assistance with manuscript submission.

Many provisions of the Affordable Care Act (ACA) have served to support the hospitalized patient. The expansion of Medicaid and the creation of state and federal insurance exchanges for the individual insurance market both significantly lessened the financial burden of hospital care for millions of Americans. Other aspects have proven more controversial, as many of the ACA’s health policy interventions linked to cost and quality in new ways, implementing untested concepts derived from healthcare services research on a national scale.

The Hospital Readmissions Reduction Program (HRRP) was no exception. Based on early research examining readmissions,¹ the ACA included a mandate for the Centers for Medicare and Medicaid Services (CMS) to establish the HRRP. Beginning in Fiscal Year 2013, the HRRP reduced payments for excessive, 30-day, risk-standardized readmissions covering six conditions and procedures. As the third leading cause of 30-day readmissions, chronic obstructive pulmonary disease (COPD) was included in the list of designated HRRP conditions.

This inclusion of COPD in HRRP was not without controversy; analysis of Medicare data from before the ACA’s implementation demonstrated that only half of all readmissions for acute exacerbations of COPD were respiratory-related and only a third were directly related to COPD.² Unsurprisingly, the high proportion of readmissions due to non-COPD-related causes is considered to be one of the leading factors for the failure of COPD readmission reduction programs to find significant reductions in readmissions.³ In this month’s issue of the Journal of Hospital Medicine, Buhr and colleagues explore differential readmission diagnoses following acute exacerbations of COPD using a validated, national, all-payer database.⁴

Like many analyses of payer datasets, this study has several limitations. First, although a large area of the US was included, the data did not include all US states. Further, as the study used multiple cross-sectional data using pooling techniques, it was not truly a longitudinal study. It was additionally limited to 10 months out of the calendar year, missing December and January, which have a high seasonal prevalence of viral respiratory illness. Finally, due to the nature of the data, COPD diagnoses were identified through administrative data known to be highly unreliable for fully capturing admissions for acute exacerbation of COPD.

Despite these limitations, the analysis by Buhr and colleagues provides additional value. They found an overall readmission rate of 17%, with just under half (7.69%) due to recurrent COPD. Patients with COPD-related readmissions were younger, had a higher proportion with Medicaid as the payer, were more frequently discharged home without services, had a shorter length of stay, and had fewer comorbidities.

Most critically, Buhr and colleagues—with a multipayer database—confirmed what researchers found in uni-payer⁵ and site-specific⁶ datasets: over half of readmissions are due to diagnoses other than COPD or respiratory-related causes. Patients readmitted due to other, unrelated diagnoses had a higher mean Elixhauser Comorbidity Index score along with higher rates of congestive heart failure and renal failure. To the practicing hospitalist, this finding supports what our internal clinical voice tells us: sicker patients are readmitted more often and more frequently with conditions unrelated to their index admission diagnosis.

The reaffirmation of the finding that the majority of readmissions are due to nonrespiratory-related causes suggests that perhaps we have a different problem than physicians and policymakers originally thought when adding COPD to the HRRP. Many COPD patients suffer from a polychronic disease, requiring a more holistic approach rather than a traditional, disease-driven, siloed approach focused solely on improving COPD-related care. It may also be true that for other subpopulations of patients with COPD, additional in-hospital and transition of care interventions are required to address patients’ multimorbidity and social determinants of health.

As physicians on the front lines of the readmitted patient, hospitalists are uniquely situated to see the challenges of populations with increasing disease complexity and disease combinations.⁷ The HRRP policy remains controversial. This is due in large part to recent work suggesting that while the HRRP may have helped reduce readmissions, its implementation may have driven the unintended consequence of increased mortality.⁸ Thus, our profession faces an existential challenge to traditional care delivery models targeting diseases. What has not been well parsed by the hospital industry or policymakers is what to do about it.

Readmission of the multimorbid patient, coupled with the challenges of the HRRP, focuses our attention on the need to transition care delivery to a model that is better suited to our patients’ needs: mass-customized, mass-produced service delivery. As physicians, we know that care delivery must be oriented around patients who have many diseases and unique life circumstances. It is our profession’s greatest challenge to collaborate with researchers and administrators to help do this with scale.

Acknowledgments

The authors thank Mary Akel for her assistance with manuscript submission.

From Tufts Medical Center, Boston, MA (Dr. Trautwein and Dr. Schwartz contributed equally to this article).

Abstract

• Objective: To examine whether an institution- and service-specific readmission prediction instrument has improved performance compared to universal tools.
• Design: Retrospective cohort study.
• Setting: Academic medical center located in Boston, MA.
• Participants: Adult inpatients admitted to a medicine service.
• Measurements: Patient attributes, inpatient service assignment, and 30-day readmission rate.
• Results: Of 7972 index admissions, 12.6% were readmitted within 30 days. Thirty-day readmissions were associated with number of medications on admission (adjusted odds ratio [OR], 1.34; 95% confidence interval [CI], 1.11-1.61) for ≥ 11 compared with ≤ 5 medications; prior admission or overnight observation (OR, 1.89; 95% CI, 1.61-2.23); and discharge (OR, 2.45; 95% CI, 1.97-3.06) to an acute care facility compared to home without services. The subspecialty services with the highest risk of readmission were bone marrow transplant/hematology (OR, 2.46; 95% CI, 1.78-3.40) and oncology (OR, 2.26; 95% CI, 1.67-3.05), as compared with general medicine/geriatrics. The C statistic for the derivation cohort was 0.67, as compared with a C statistic of 0.63 for the LACE index.
• Conclusion: A hospital service–specific 30-day readmission prediction tool showed incrementally improved performance over the widely used LACE index.

Keywords: rehospitalization; quality of care; predictive model; hospital medicine.

Readmissions are a costly problem in the United States. The readmission rate among Medicare beneficiaries aged 65 years and older was 17.1 per 100 live discharges in 2016.¹ Although both the relationship between readmissions and quality of care and the use of readmissions as a quality measure are contested,² efforts to reduce potentially avoidable rehospitalizations have received widespread attention and readmissions are a focus of reimbursement reform.^3-5

The LACE index is a widely used model for readmission risk prediction.⁶ Derived from a study of hospitalized patients in Ontario, Canada, LACE uses information about length of stay, acuity, comorbidities, and emergency department utilization to estimate readmission risk. Models such as LACE+, LACE-rt, and HOSPITAL have tried to improve on LACE’s performance, but models with strong discriminative ability are lacking.^7-9 Institution-specific models exist,¹⁰ as do well-organized multicenter studies,⁷ but their generalizability is limited due to population differences or inclusion of data difficult to extract from patients in real-time, such as data gathered through a socioeconomic questionnaire.

Given hospital-specific differences in operational performance and patient population, we sought to develop a statistical model for 30-day readmission prediction and demonstrate the process that could be utilized by other institutions to identify high-risk patients for intensive case management and discharge planning.

Methods

Study Design

We conducted a retrospective cohort study of all admissions to the medicine service at a 415-bed teaching hospital in Boston, MA, from September 1, 2013, through August 31, 2016. Patients are admitted through the emergency department or directly admitted from hospital-based practices or a statewide network of private practices.

Data Collection

Data were abstracted from electronic medical and billing records from the first (index) admission for each patient during the study period. Thirty-day readmission was defined as an unplanned admission in the 30 days following the index discharge date. We excluded patients readmitted after leaving against medical advice and planned readmissions based on information in discharge summaries.

The study team identified candidate risk factors by referencing related published research and with input from a multidisciplinary task force charged with developing strategies to reduce 30-day readmissions. Task force members included attending and resident physicians, pharmacists, nurses, case managers, and administrators. The task force considered factors that could be extracted from the electronic medical record, including demographics, location of care, and clinical measures such as diagnostic codes, as well as data available in nursing, social work, and case management notes. Decisions regarding potential risk factors were reached within the group based on institutional experience, availability, and quality of data within the electronic record for specific variables, as well as published research on the subject, with the goal of selecting variables that could be easily identified before discharge and used to generate a predictive score for use in discharge planning.

Variables

Variables initially considered for inclusion in univariate analyses included demographic characteristics of age, gender, and a combined race/ethnicity variable, delineated as either non-Hispanic white, non-Hispanic black, Asian/Asian Indian, or Hispanic. Those with race listed as other or missing were set to missing. Primary language was categorized as English versus non-English. We included a number of variables related to the severity of the patient’s medical condition during the index admission, including any stay in an intensive care unit (ICU) and number of medications on admission, divided into 3 groups, 0-5, 6-10, and 11 or more. We also included separate indicators for admissions on warfarin or chronic opioids. Charlson comorbidity score as well as heart failure, diabetes, and chronic obstructive pulmonary disease were included as separate variables, since these specific diagnoses have high comorbidity and risk of readmission.

Because the hospital’s medicine service is divided into subspecialty services, we included the admitting service and discharge unit to assess whether certain teams or units were associated with readmission. Discharge disposition was categorized as home with services (ie, physical therapy and visiting nurse), home without services, skilled nursing facility, acute care facility, or other. We included a variable to assess patient frailty and mobility based on the presence of a physical therapy consult. We incorporated social determinants of health, including insurance coverage (private insurance, Medicare, Medicaid, subsidized, or uninsured); per capita income from the patient’s zip code as a proxy for economic status (divided into quartiles for analysis); and substance abuse and alcohol abuse (based on International Classification of Diseases, 10th revision codes). We considered whether the discharge was on a weekday or weekend, and considered distance to the hospital in relation to Boston, either within route 128 (roughly within 15-20 miles of the medical center), within interstate 495 (roughly within 30-40 miles of the medical center), or beyond this. We considered but were unable to incorporate candidate variables that had inconsistent availability in the electronic medical record, such as the Braden score, level of independence with activities of daily living, nursing-determined fall risk, presence of a social work or nutrition consultation, CAGE questionnaire for alcohol abuse, delirium assessment score, the number of adults living in the home, the number of dependents, and marital status.

Analysis

We created a derivation cohort using admission data from September 1, 2013, through November 30, 2015. We used a backward selection process to include variables in the derivation model. Any variable associated with 30-day readmissions with a P value < 0.10 in univariate analyses was considered as a candidate variable. To be retained in the multivariable model, each variable was required to have a significant association with 30-day readmission at the P < 0.05 level. We used beta coefficients to create a numerical score reflective of probability of readmission.

We then created a validation cohort using admissions data between December 1, 2015, and August 31, 2016. We applied the scoring algorithm from the derivation cohort to the validation cohort and compared the discriminative ability of the 2 models using the area under the receiver operating characteristic (ROC) curve. We also compared the area under the ROC curve of our predictive model to the LACE index using the nonparametric approach of DeLong and colleagues.^8,11

Results

The derivation cohort consisted of 7972 index admissions, of which 12.6% were readmitted within 30 days. The patient population was 45% female, 70% white, and 85% English-speaking, with an average age of 61.4 years (standard deviation, 18.1, Table 1). Most patients had either private insurance (43%) or Medicare (41%).

Many patients were medically complex: 21% required ICU care, 29% were taking 11 or more medications on admission, and 52% had a Charlson score of 2 or more. In the previous 6 months, 14% had an emergency department visit and 16% had an admission or overnight observation. The rate of drug or alcohol abuse was 13%. The majority of patients were discharged home without services (54%), while 23% were discharged home with services, 13% were discharged to a skilled nursing facility, and 9% were discharged to an acute care facility.

Factors Associated With 30-Day Readmission

Table 2 displays the univariate and multivariate associations with 30-day readmissions in the derivation cohort. Variables significantly associated with readmission in univariate analysis were Charlson comorbidity score, history of diabetes, ICU utilization, previous emergency department visit, inpatient or observation stay within 6 months, number of medications on admission, use of opioids on admission, a diagnosis of diabetes, type of insurance, physical therapy consultation, admitting service, and discharge disposition. Variables not significant in univariate analysis included warfarin use, alcohol/drug abuse, distance from the hospital, and demographic variables (age, sex, race, language, income by zip code).

Variables associated with readmission in the final multivariate analysis model included a Charlson score of 2 or higher (compared to a score of 0; odds ratio [OR], 1.36; 95% confidence interval [CI], 1.11-1.66); any ICU stay (OR, 1.29; 95% CI, 1.08-1.53); number of medications on admission (OR, 1.34; 95% CI, 1.11-1.61) for 11 or more compared with 5 or fewer medications; prior admission or overnight observation (OR, 1.89; 95% CI, 1.61-2.23); and disposition on discharge to an acute care facility (OR, 2.45; 95% CI, 1.96-3.06), skilled nursing facility (OR, 1.53; 95% CI, 1.23-1.89), or home with services (OR, 1.64; 95% CI, 1.38-1.94) compared with home discharge without services. The hospital service from which the patient was discharged was significantly associated with readmission; the subspecialty services with the highest odds ratios were bone marrow transplant/hematology (OR, 2.46; 95% CI, 1.77-3.40) and oncology (OR, 2.26; 95% CI, 1.67-3.05), as compared with general medicine/geriatrics.

Model Derivation and Validation

We utilized the beta coefficients from the multivariate analysis to create a scoring tool to predict the likelihood of 30-day readmission. We rounded each beta coefficient and calculated a readmission score by adding together the rounded beta coefficients of each of the significant variables. Table 3 presents the cumulative percentage of discharges at each score level, as well as the calculated cumulative percentage of potential readmissions. For example, in our population, a score of 6 or greater accounted for 18% of all discharges, but 36% of all 30-day readmissions.

The ROC curves for the derivation model and LACE index are shown in the Figure. The C statistic for the derivation cohort was 0.67, as compared with a C statistic of 0.63 for the calculated LACE index (P < 0.0001). The validation cohort had a C statistic similar to that of the derivation cohort (0.66).

Discussion

We developed a predictive model that can be used during admission to stratify patients for intensive case management and discharge planning. The model included Charlson score, ICU utilization, admission to inpatient services or observation, visits to the ED in the past 6 months, number of medications on admission, hospital service, and discharge disposition. The C statistic of 0.67 is better than that of the LACE predictive model for our population, although both reflect only modest predictive value.

While our model, which was developed and validated at a single institution, may not be generalizable to other institutions, the method of developing a readmission risk prediction model specific to an institution is readily replicable. While standardized tools for predicting readmission risk exist, they do not necessarily account for unique patient populations and medical complexity at individual institutions. We examined patients discharged over 3 years from the medicine services, creating a service-specific model. Our approach could lead to the widespread development of service- and institution-specific models that take into account the risks and resources appropriate to each patient population and setting.

Many of the factors included in our model were indicative of the patients’ medical complexity, with Charlson comorbidity score and the number of discharge medications strongly associated with readmission. In the derivation of the LACE model, many patients were middle-aged and independent in their activities of daily living, and more than 75% had a Charlson comorbidity score of 0.6.⁶ In our population, by contrast, the majority of patients had a Charlson score of 2 or greater. Health care utilization was a strong positive predictor in both our model and in LACE. The finding that the Charlson comorbidity score was a better predictor than any single chronic illness suggests that medical complexity and comorbidities increase the likelihood of admission more than any 1 chronic condition. Our model incorporates discharge disposition in readmission prediction, another factor associated with medical complexity and frailty.

A surprising finding in our study was the lack of association between social determinants of health, such as alcohol and drug abuse, and readmission risk. We posit several reasons that may account for this finding. First, the population served by the medical center may be too small or homogeneous with respect to social determinants of health to detect a difference in readmission risk. Second, markers available in the electronic medical record to determine social needs may be too crude to distinguish degrees of vulnerability that increase the risk of readmission. We do not discount the importance of social determinants of health as predictors of readmission risk, but we do acknowledge the limitations of the data incorporated in our model.

Predictive models are useful only if they can be incorporated into workflow to identify high-risk patients. Prior to developing and using our model, we used LACE inconsistently because it required length of stay as 1 of the variables. Because the variables in our model are collected and recorded routinely at admission in our electronic medical record, the readmission risk score is calculated and displayed in a daily high-risk patient report. This automated process has afforded a more consistent and reliable approach to readmission risk assessment than previous efforts to assess the LACE index. Case managers use the high-risk patient report to identify patients who require enhanced care coordination and discharge planning. Since the introduction of this predictive model, we have noted a 10% reduction in the hospital’s 30-day readmission rate.

This project was subject to several limitations. Because data on admissions to other facilities were unavailable, we may have underestimated the risk of readmission to other facilities. Our results may not be generalizable to other organizations, although we believe that the methods are readily replicable. The performance of the model and its replication with a validation cohort are strengths of the approach.

Conclusion

We created a hospital service–specific 30-day readmission prediction tool whose performance improved incrementally over the widely used LACE index. This research suggests that readmission prediction is highly context-specific and that organizations would do well to examine the readmission risk factors most pertinent to the populations they serve. We believe that “customized” readmission risk prediction models for particular services in specific hospitals may offer a superior method to identify high-risk patients who may benefit from individualized care planning. Future research is needed to understand how best to capture information about the attributes of vulnerable populations, so that this information can be incorporated into future risk models.

Corresponding author: Karen Freund, MD, Tufts Medical Center, 800 Washington St., Boston, MA 02111; [email protected].

Financial disclosures: None.

Funding for this work was provided by the Commonwealth of Massachusetts, Executive Office of Health and Human Services.

From Tufts Medical Center, Boston, MA (Dr. Trautwein and Dr. Schwartz contributed equally to this article).

Abstract

• Objective: To examine whether an institution- and service-specific readmission prediction instrument has improved performance compared to universal tools.
• Design: Retrospective cohort study.
• Setting: Academic medical center located in Boston, MA.
• Participants: Adult inpatients admitted to a medicine service.
• Measurements: Patient attributes, inpatient service assignment, and 30-day readmission rate.
• Results: Of 7972 index admissions, 12.6% were readmitted within 30 days. Thirty-day readmissions were associated with number of medications on admission (adjusted odds ratio [OR], 1.34; 95% confidence interval [CI], 1.11-1.61) for ≥ 11 compared with ≤ 5 medications; prior admission or overnight observation (OR, 1.89; 95% CI, 1.61-2.23); and discharge (OR, 2.45; 95% CI, 1.97-3.06) to an acute care facility compared to home without services. The subspecialty services with the highest risk of readmission were bone marrow transplant/hematology (OR, 2.46; 95% CI, 1.78-3.40) and oncology (OR, 2.26; 95% CI, 1.67-3.05), as compared with general medicine/geriatrics. The C statistic for the derivation cohort was 0.67, as compared with a C statistic of 0.63 for the LACE index.
• Conclusion: A hospital service–specific 30-day readmission prediction tool showed incrementally improved performance over the widely used LACE index.

Keywords: rehospitalization; quality of care; predictive model; hospital medicine.

Readmissions are a costly problem in the United States. The readmission rate among Medicare beneficiaries aged 65 years and older was 17.1 per 100 live discharges in 2016.¹ Although both the relationship between readmissions and quality of care and the use of readmissions as a quality measure are contested,² efforts to reduce potentially avoidable rehospitalizations have received widespread attention and readmissions are a focus of reimbursement reform.^3-5

The LACE index is a widely used model for readmission risk prediction.⁶ Derived from a study of hospitalized patients in Ontario, Canada, LACE uses information about length of stay, acuity, comorbidities, and emergency department utilization to estimate readmission risk. Models such as LACE+, LACE-rt, and HOSPITAL have tried to improve on LACE’s performance, but models with strong discriminative ability are lacking.^7-9 Institution-specific models exist,¹⁰ as do well-organized multicenter studies,⁷ but their generalizability is limited due to population differences or inclusion of data difficult to extract from patients in real-time, such as data gathered through a socioeconomic questionnaire.

Given hospital-specific differences in operational performance and patient population, we sought to develop a statistical model for 30-day readmission prediction and demonstrate the process that could be utilized by other institutions to identify high-risk patients for intensive case management and discharge planning.

Methods

Study Design

We conducted a retrospective cohort study of all admissions to the medicine service at a 415-bed teaching hospital in Boston, MA, from September 1, 2013, through August 31, 2016. Patients are admitted through the emergency department or directly admitted from hospital-based practices or a statewide network of private practices.

Data Collection

Data were abstracted from electronic medical and billing records from the first (index) admission for each patient during the study period. Thirty-day readmission was defined as an unplanned admission in the 30 days following the index discharge date. We excluded patients readmitted after leaving against medical advice and planned readmissions based on information in discharge summaries.

The study team identified candidate risk factors by referencing related published research and with input from a multidisciplinary task force charged with developing strategies to reduce 30-day readmissions. Task force members included attending and resident physicians, pharmacists, nurses, case managers, and administrators. The task force considered factors that could be extracted from the electronic medical record, including demographics, location of care, and clinical measures such as diagnostic codes, as well as data available in nursing, social work, and case management notes. Decisions regarding potential risk factors were reached within the group based on institutional experience, availability, and quality of data within the electronic record for specific variables, as well as published research on the subject, with the goal of selecting variables that could be easily identified before discharge and used to generate a predictive score for use in discharge planning.

Variables

Variables initially considered for inclusion in univariate analyses included demographic characteristics of age, gender, and a combined race/ethnicity variable, delineated as either non-Hispanic white, non-Hispanic black, Asian/Asian Indian, or Hispanic. Those with race listed as other or missing were set to missing. Primary language was categorized as English versus non-English. We included a number of variables related to the severity of the patient’s medical condition during the index admission, including any stay in an intensive care unit (ICU) and number of medications on admission, divided into 3 groups, 0-5, 6-10, and 11 or more. We also included separate indicators for admissions on warfarin or chronic opioids. Charlson comorbidity score as well as heart failure, diabetes, and chronic obstructive pulmonary disease were included as separate variables, since these specific diagnoses have high comorbidity and risk of readmission.

Because the hospital’s medicine service is divided into subspecialty services, we included the admitting service and discharge unit to assess whether certain teams or units were associated with readmission. Discharge disposition was categorized as home with services (ie, physical therapy and visiting nurse), home without services, skilled nursing facility, acute care facility, or other. We included a variable to assess patient frailty and mobility based on the presence of a physical therapy consult. We incorporated social determinants of health, including insurance coverage (private insurance, Medicare, Medicaid, subsidized, or uninsured); per capita income from the patient’s zip code as a proxy for economic status (divided into quartiles for analysis); and substance abuse and alcohol abuse (based on International Classification of Diseases, 10th revision codes). We considered whether the discharge was on a weekday or weekend, and considered distance to the hospital in relation to Boston, either within route 128 (roughly within 15-20 miles of the medical center), within interstate 495 (roughly within 30-40 miles of the medical center), or beyond this. We considered but were unable to incorporate candidate variables that had inconsistent availability in the electronic medical record, such as the Braden score, level of independence with activities of daily living, nursing-determined fall risk, presence of a social work or nutrition consultation, CAGE questionnaire for alcohol abuse, delirium assessment score, the number of adults living in the home, the number of dependents, and marital status.

Analysis

We created a derivation cohort using admission data from September 1, 2013, through November 30, 2015. We used a backward selection process to include variables in the derivation model. Any variable associated with 30-day readmissions with a P value < 0.10 in univariate analyses was considered as a candidate variable. To be retained in the multivariable model, each variable was required to have a significant association with 30-day readmission at the P < 0.05 level. We used beta coefficients to create a numerical score reflective of probability of readmission.

We then created a validation cohort using admissions data between December 1, 2015, and August 31, 2016. We applied the scoring algorithm from the derivation cohort to the validation cohort and compared the discriminative ability of the 2 models using the area under the receiver operating characteristic (ROC) curve. We also compared the area under the ROC curve of our predictive model to the LACE index using the nonparametric approach of DeLong and colleagues.^8,11

Results

The derivation cohort consisted of 7972 index admissions, of which 12.6% were readmitted within 30 days. The patient population was 45% female, 70% white, and 85% English-speaking, with an average age of 61.4 years (standard deviation, 18.1, Table 1). Most patients had either private insurance (43%) or Medicare (41%).

Many patients were medically complex: 21% required ICU care, 29% were taking 11 or more medications on admission, and 52% had a Charlson score of 2 or more. In the previous 6 months, 14% had an emergency department visit and 16% had an admission or overnight observation. The rate of drug or alcohol abuse was 13%. The majority of patients were discharged home without services (54%), while 23% were discharged home with services, 13% were discharged to a skilled nursing facility, and 9% were discharged to an acute care facility.

Factors Associated With 30-Day Readmission

Table 2 displays the univariate and multivariate associations with 30-day readmissions in the derivation cohort. Variables significantly associated with readmission in univariate analysis were Charlson comorbidity score, history of diabetes, ICU utilization, previous emergency department visit, inpatient or observation stay within 6 months, number of medications on admission, use of opioids on admission, a diagnosis of diabetes, type of insurance, physical therapy consultation, admitting service, and discharge disposition. Variables not significant in univariate analysis included warfarin use, alcohol/drug abuse, distance from the hospital, and demographic variables (age, sex, race, language, income by zip code).

Variables associated with readmission in the final multivariate analysis model included a Charlson score of 2 or higher (compared to a score of 0; odds ratio [OR], 1.36; 95% confidence interval [CI], 1.11-1.66); any ICU stay (OR, 1.29; 95% CI, 1.08-1.53); number of medications on admission (OR, 1.34; 95% CI, 1.11-1.61) for 11 or more compared with 5 or fewer medications; prior admission or overnight observation (OR, 1.89; 95% CI, 1.61-2.23); and disposition on discharge to an acute care facility (OR, 2.45; 95% CI, 1.96-3.06), skilled nursing facility (OR, 1.53; 95% CI, 1.23-1.89), or home with services (OR, 1.64; 95% CI, 1.38-1.94) compared with home discharge without services. The hospital service from which the patient was discharged was significantly associated with readmission; the subspecialty services with the highest odds ratios were bone marrow transplant/hematology (OR, 2.46; 95% CI, 1.77-3.40) and oncology (OR, 2.26; 95% CI, 1.67-3.05), as compared with general medicine/geriatrics.

Model Derivation and Validation

We utilized the beta coefficients from the multivariate analysis to create a scoring tool to predict the likelihood of 30-day readmission. We rounded each beta coefficient and calculated a readmission score by adding together the rounded beta coefficients of each of the significant variables. Table 3 presents the cumulative percentage of discharges at each score level, as well as the calculated cumulative percentage of potential readmissions. For example, in our population, a score of 6 or greater accounted for 18% of all discharges, but 36% of all 30-day readmissions.

The ROC curves for the derivation model and LACE index are shown in the Figure. The C statistic for the derivation cohort was 0.67, as compared with a C statistic of 0.63 for the calculated LACE index (P < 0.0001). The validation cohort had a C statistic similar to that of the derivation cohort (0.66).

Discussion

We developed a predictive model that can be used during admission to stratify patients for intensive case management and discharge planning. The model included Charlson score, ICU utilization, admission to inpatient services or observation, visits to the ED in the past 6 months, number of medications on admission, hospital service, and discharge disposition. The C statistic of 0.67 is better than that of the LACE predictive model for our population, although both reflect only modest predictive value.

While our model, which was developed and validated at a single institution, may not be generalizable to other institutions, the method of developing a readmission risk prediction model specific to an institution is readily replicable. While standardized tools for predicting readmission risk exist, they do not necessarily account for unique patient populations and medical complexity at individual institutions. We examined patients discharged over 3 years from the medicine services, creating a service-specific model. Our approach could lead to the widespread development of service- and institution-specific models that take into account the risks and resources appropriate to each patient population and setting.

Many of the factors included in our model were indicative of the patients’ medical complexity, with Charlson comorbidity score and the number of discharge medications strongly associated with readmission. In the derivation of the LACE model, many patients were middle-aged and independent in their activities of daily living, and more than 75% had a Charlson comorbidity score of 0.6.⁶ In our population, by contrast, the majority of patients had a Charlson score of 2 or greater. Health care utilization was a strong positive predictor in both our model and in LACE. The finding that the Charlson comorbidity score was a better predictor than any single chronic illness suggests that medical complexity and comorbidities increase the likelihood of admission more than any 1 chronic condition. Our model incorporates discharge disposition in readmission prediction, another factor associated with medical complexity and frailty.

A surprising finding in our study was the lack of association between social determinants of health, such as alcohol and drug abuse, and readmission risk. We posit several reasons that may account for this finding. First, the population served by the medical center may be too small or homogeneous with respect to social determinants of health to detect a difference in readmission risk. Second, markers available in the electronic medical record to determine social needs may be too crude to distinguish degrees of vulnerability that increase the risk of readmission. We do not discount the importance of social determinants of health as predictors of readmission risk, but we do acknowledge the limitations of the data incorporated in our model.

Predictive models are useful only if they can be incorporated into workflow to identify high-risk patients. Prior to developing and using our model, we used LACE inconsistently because it required length of stay as 1 of the variables. Because the variables in our model are collected and recorded routinely at admission in our electronic medical record, the readmission risk score is calculated and displayed in a daily high-risk patient report. This automated process has afforded a more consistent and reliable approach to readmission risk assessment than previous efforts to assess the LACE index. Case managers use the high-risk patient report to identify patients who require enhanced care coordination and discharge planning. Since the introduction of this predictive model, we have noted a 10% reduction in the hospital’s 30-day readmission rate.

This project was subject to several limitations. Because data on admissions to other facilities were unavailable, we may have underestimated the risk of readmission to other facilities. Our results may not be generalizable to other organizations, although we believe that the methods are readily replicable. The performance of the model and its replication with a validation cohort are strengths of the approach.

Conclusion

We created a hospital service–specific 30-day readmission prediction tool whose performance improved incrementally over the widely used LACE index. This research suggests that readmission prediction is highly context-specific and that organizations would do well to examine the readmission risk factors most pertinent to the populations they serve. We believe that “customized” readmission risk prediction models for particular services in specific hospitals may offer a superior method to identify high-risk patients who may benefit from individualized care planning. Future research is needed to understand how best to capture information about the attributes of vulnerable populations, so that this information can be incorporated into future risk models.

Corresponding author: Karen Freund, MD, Tufts Medical Center, 800 Washington St., Boston, MA 02111; [email protected].

Financial disclosures: None.

Funding for this work was provided by the Commonwealth of Massachusetts, Executive Office of Health and Human Services.

From Tufts Medical Center, Boston, MA (Dr. Trautwein and Dr. Schwartz contributed equally to this article).

Abstract

• Objective: To examine whether an institution- and service-specific readmission prediction instrument has improved performance compared to universal tools.
• Design: Retrospective cohort study.
• Setting: Academic medical center located in Boston, MA.
• Participants: Adult inpatients admitted to a medicine service.
• Measurements: Patient attributes, inpatient service assignment, and 30-day readmission rate.
• Results: Of 7972 index admissions, 12.6% were readmitted within 30 days. Thirty-day readmissions were associated with number of medications on admission (adjusted odds ratio [OR], 1.34; 95% confidence interval [CI], 1.11-1.61) for ≥ 11 compared with ≤ 5 medications; prior admission or overnight observation (OR, 1.89; 95% CI, 1.61-2.23); and discharge (OR, 2.45; 95% CI, 1.97-3.06) to an acute care facility compared to home without services. The subspecialty services with the highest risk of readmission were bone marrow transplant/hematology (OR, 2.46; 95% CI, 1.78-3.40) and oncology (OR, 2.26; 95% CI, 1.67-3.05), as compared with general medicine/geriatrics. The C statistic for the derivation cohort was 0.67, as compared with a C statistic of 0.63 for the LACE index.
• Conclusion: A hospital service–specific 30-day readmission prediction tool showed incrementally improved performance over the widely used LACE index.

Keywords: rehospitalization; quality of care; predictive model; hospital medicine.

Readmissions are a costly problem in the United States. The readmission rate among Medicare beneficiaries aged 65 years and older was 17.1 per 100 live discharges in 2016.¹ Although both the relationship between readmissions and quality of care and the use of readmissions as a quality measure are contested,² efforts to reduce potentially avoidable rehospitalizations have received widespread attention and readmissions are a focus of reimbursement reform.^3-5

The LACE index is a widely used model for readmission risk prediction.⁶ Derived from a study of hospitalized patients in Ontario, Canada, LACE uses information about length of stay, acuity, comorbidities, and emergency department utilization to estimate readmission risk. Models such as LACE+, LACE-rt, and HOSPITAL have tried to improve on LACE’s performance, but models with strong discriminative ability are lacking.^7-9 Institution-specific models exist,¹⁰ as do well-organized multicenter studies,⁷ but their generalizability is limited due to population differences or inclusion of data difficult to extract from patients in real-time, such as data gathered through a socioeconomic questionnaire.

Given hospital-specific differences in operational performance and patient population, we sought to develop a statistical model for 30-day readmission prediction and demonstrate the process that could be utilized by other institutions to identify high-risk patients for intensive case management and discharge planning.

Methods

Study Design

We conducted a retrospective cohort study of all admissions to the medicine service at a 415-bed teaching hospital in Boston, MA, from September 1, 2013, through August 31, 2016. Patients are admitted through the emergency department or directly admitted from hospital-based practices or a statewide network of private practices.

Data Collection

Data were abstracted from electronic medical and billing records from the first (index) admission for each patient during the study period. Thirty-day readmission was defined as an unplanned admission in the 30 days following the index discharge date. We excluded patients readmitted after leaving against medical advice and planned readmissions based on information in discharge summaries.

The study team identified candidate risk factors by referencing related published research and with input from a multidisciplinary task force charged with developing strategies to reduce 30-day readmissions. Task force members included attending and resident physicians, pharmacists, nurses, case managers, and administrators. The task force considered factors that could be extracted from the electronic medical record, including demographics, location of care, and clinical measures such as diagnostic codes, as well as data available in nursing, social work, and case management notes. Decisions regarding potential risk factors were reached within the group based on institutional experience, availability, and quality of data within the electronic record for specific variables, as well as published research on the subject, with the goal of selecting variables that could be easily identified before discharge and used to generate a predictive score for use in discharge planning.

Variables

Variables initially considered for inclusion in univariate analyses included demographic characteristics of age, gender, and a combined race/ethnicity variable, delineated as either non-Hispanic white, non-Hispanic black, Asian/Asian Indian, or Hispanic. Those with race listed as other or missing were set to missing. Primary language was categorized as English versus non-English. We included a number of variables related to the severity of the patient’s medical condition during the index admission, including any stay in an intensive care unit (ICU) and number of medications on admission, divided into 3 groups, 0-5, 6-10, and 11 or more. We also included separate indicators for admissions on warfarin or chronic opioids. Charlson comorbidity score as well as heart failure, diabetes, and chronic obstructive pulmonary disease were included as separate variables, since these specific diagnoses have high comorbidity and risk of readmission.

Because the hospital’s medicine service is divided into subspecialty services, we included the admitting service and discharge unit to assess whether certain teams or units were associated with readmission. Discharge disposition was categorized as home with services (ie, physical therapy and visiting nurse), home without services, skilled nursing facility, acute care facility, or other. We included a variable to assess patient frailty and mobility based on the presence of a physical therapy consult. We incorporated social determinants of health, including insurance coverage (private insurance, Medicare, Medicaid, subsidized, or uninsured); per capita income from the patient’s zip code as a proxy for economic status (divided into quartiles for analysis); and substance abuse and alcohol abuse (based on International Classification of Diseases, 10th revision codes). We considered whether the discharge was on a weekday or weekend, and considered distance to the hospital in relation to Boston, either within route 128 (roughly within 15-20 miles of the medical center), within interstate 495 (roughly within 30-40 miles of the medical center), or beyond this. We considered but were unable to incorporate candidate variables that had inconsistent availability in the electronic medical record, such as the Braden score, level of independence with activities of daily living, nursing-determined fall risk, presence of a social work or nutrition consultation, CAGE questionnaire for alcohol abuse, delirium assessment score, the number of adults living in the home, the number of dependents, and marital status.

Analysis

We created a derivation cohort using admission data from September 1, 2013, through November 30, 2015. We used a backward selection process to include variables in the derivation model. Any variable associated with 30-day readmissions with a P value < 0.10 in univariate analyses was considered as a candidate variable. To be retained in the multivariable model, each variable was required to have a significant association with 30-day readmission at the P < 0.05 level. We used beta coefficients to create a numerical score reflective of probability of readmission.

We then created a validation cohort using admissions data between December 1, 2015, and August 31, 2016. We applied the scoring algorithm from the derivation cohort to the validation cohort and compared the discriminative ability of the 2 models using the area under the receiver operating characteristic (ROC) curve. We also compared the area under the ROC curve of our predictive model to the LACE index using the nonparametric approach of DeLong and colleagues.^8,11

Results

The derivation cohort consisted of 7972 index admissions, of which 12.6% were readmitted within 30 days. The patient population was 45% female, 70% white, and 85% English-speaking, with an average age of 61.4 years (standard deviation, 18.1, Table 1). Most patients had either private insurance (43%) or Medicare (41%).

Many patients were medically complex: 21% required ICU care, 29% were taking 11 or more medications on admission, and 52% had a Charlson score of 2 or more. In the previous 6 months, 14% had an emergency department visit and 16% had an admission or overnight observation. The rate of drug or alcohol abuse was 13%. The majority of patients were discharged home without services (54%), while 23% were discharged home with services, 13% were discharged to a skilled nursing facility, and 9% were discharged to an acute care facility.

Factors Associated With 30-Day Readmission

Table 2 displays the univariate and multivariate associations with 30-day readmissions in the derivation cohort. Variables significantly associated with readmission in univariate analysis were Charlson comorbidity score, history of diabetes, ICU utilization, previous emergency department visit, inpatient or observation stay within 6 months, number of medications on admission, use of opioids on admission, a diagnosis of diabetes, type of insurance, physical therapy consultation, admitting service, and discharge disposition. Variables not significant in univariate analysis included warfarin use, alcohol/drug abuse, distance from the hospital, and demographic variables (age, sex, race, language, income by zip code).

Variables associated with readmission in the final multivariate analysis model included a Charlson score of 2 or higher (compared to a score of 0; odds ratio [OR], 1.36; 95% confidence interval [CI], 1.11-1.66); any ICU stay (OR, 1.29; 95% CI, 1.08-1.53); number of medications on admission (OR, 1.34; 95% CI, 1.11-1.61) for 11 or more compared with 5 or fewer medications; prior admission or overnight observation (OR, 1.89; 95% CI, 1.61-2.23); and disposition on discharge to an acute care facility (OR, 2.45; 95% CI, 1.96-3.06), skilled nursing facility (OR, 1.53; 95% CI, 1.23-1.89), or home with services (OR, 1.64; 95% CI, 1.38-1.94) compared with home discharge without services. The hospital service from which the patient was discharged was significantly associated with readmission; the subspecialty services with the highest odds ratios were bone marrow transplant/hematology (OR, 2.46; 95% CI, 1.77-3.40) and oncology (OR, 2.26; 95% CI, 1.67-3.05), as compared with general medicine/geriatrics.

Model Derivation and Validation

We utilized the beta coefficients from the multivariate analysis to create a scoring tool to predict the likelihood of 30-day readmission. We rounded each beta coefficient and calculated a readmission score by adding together the rounded beta coefficients of each of the significant variables. Table 3 presents the cumulative percentage of discharges at each score level, as well as the calculated cumulative percentage of potential readmissions. For example, in our population, a score of 6 or greater accounted for 18% of all discharges, but 36% of all 30-day readmissions.

The ROC curves for the derivation model and LACE index are shown in the Figure. The C statistic for the derivation cohort was 0.67, as compared with a C statistic of 0.63 for the calculated LACE index (P < 0.0001). The validation cohort had a C statistic similar to that of the derivation cohort (0.66).

Discussion

We developed a predictive model that can be used during admission to stratify patients for intensive case management and discharge planning. The model included Charlson score, ICU utilization, admission to inpatient services or observation, visits to the ED in the past 6 months, number of medications on admission, hospital service, and discharge disposition. The C statistic of 0.67 is better than that of the LACE predictive model for our population, although both reflect only modest predictive value.

While our model, which was developed and validated at a single institution, may not be generalizable to other institutions, the method of developing a readmission risk prediction model specific to an institution is readily replicable. While standardized tools for predicting readmission risk exist, they do not necessarily account for unique patient populations and medical complexity at individual institutions. We examined patients discharged over 3 years from the medicine services, creating a service-specific model. Our approach could lead to the widespread development of service- and institution-specific models that take into account the risks and resources appropriate to each patient population and setting.

Many of the factors included in our model were indicative of the patients’ medical complexity, with Charlson comorbidity score and the number of discharge medications strongly associated with readmission. In the derivation of the LACE model, many patients were middle-aged and independent in their activities of daily living, and more than 75% had a Charlson comorbidity score of 0.6.⁶ In our population, by contrast, the majority of patients had a Charlson score of 2 or greater. Health care utilization was a strong positive predictor in both our model and in LACE. The finding that the Charlson comorbidity score was a better predictor than any single chronic illness suggests that medical complexity and comorbidities increase the likelihood of admission more than any 1 chronic condition. Our model incorporates discharge disposition in readmission prediction, another factor associated with medical complexity and frailty.

A surprising finding in our study was the lack of association between social determinants of health, such as alcohol and drug abuse, and readmission risk. We posit several reasons that may account for this finding. First, the population served by the medical center may be too small or homogeneous with respect to social determinants of health to detect a difference in readmission risk. Second, markers available in the electronic medical record to determine social needs may be too crude to distinguish degrees of vulnerability that increase the risk of readmission. We do not discount the importance of social determinants of health as predictors of readmission risk, but we do acknowledge the limitations of the data incorporated in our model.

Predictive models are useful only if they can be incorporated into workflow to identify high-risk patients. Prior to developing and using our model, we used LACE inconsistently because it required length of stay as 1 of the variables. Because the variables in our model are collected and recorded routinely at admission in our electronic medical record, the readmission risk score is calculated and displayed in a daily high-risk patient report. This automated process has afforded a more consistent and reliable approach to readmission risk assessment than previous efforts to assess the LACE index. Case managers use the high-risk patient report to identify patients who require enhanced care coordination and discharge planning. Since the introduction of this predictive model, we have noted a 10% reduction in the hospital’s 30-day readmission rate.

This project was subject to several limitations. Because data on admissions to other facilities were unavailable, we may have underestimated the risk of readmission to other facilities. Our results may not be generalizable to other organizations, although we believe that the methods are readily replicable. The performance of the model and its replication with a validation cohort are strengths of the approach.

Conclusion

We created a hospital service–specific 30-day readmission prediction tool whose performance improved incrementally over the widely used LACE index. This research suggests that readmission prediction is highly context-specific and that organizations would do well to examine the readmission risk factors most pertinent to the populations they serve. We believe that “customized” readmission risk prediction models for particular services in specific hospitals may offer a superior method to identify high-risk patients who may benefit from individualized care planning. Future research is needed to understand how best to capture information about the attributes of vulnerable populations, so that this information can be incorporated into future risk models.

Corresponding author: Karen Freund, MD, Tufts Medical Center, 800 Washington St., Boston, MA 02111; [email protected].

Financial disclosures: None.

Funding for this work was provided by the Commonwealth of Massachusetts, Executive Office of Health and Human Services.

From the Institute for Patient- and Family-Centered Care, Bethesda, MD (Ms. Dokken and Ms. Johnson), and Northwell Health, New Hyde Park, NY (Dr. Barden, Ms. Tuomey, and Ms. Giammarinaro).

Abstract

Objective: To describe the growth of Better Together: Partnering with Families, a campaign launched in 2014 to eliminate restrictive hospital visiting policies and to put in place policies that recognize families as partners in care, and to discuss the processes involved in implementing the initiative in a large, integrated health system.

Methods: Descriptive report.

Results: In June 2014, the Institute for Patient- and Family-Centered Care (IPFCC) launched the Better Together campaign to emphasize the importance of family presence and participation to the quality, experience, safety, and outcomes of care. Since then, this initiative has expanded in both the United States and Canada. With support from 2 funders in the United States, special attention was focused on acute care hospitals across New York State. Nearly 50 hospitals participated in 2 separate but related projects. Fifteen of the hospitals are part of Northwell Health, New York State’s largest health system. Over a 10-month period, these hospitals made significant progress in changing policy, practice, and communication to support family presence.

Conclusion: The Better Together initiative was implemented across a health system with strong support from leadership and the involvement of patient and family advisors. An intervention offering structured training, coaching, and resources, like IPFCC’s Better Together initiative, can facilitate the change process.

Keywords: family presence; visiting policies; patient-centered care; family-centered care; patient experience.

The presence of families at the bedside of patients is often restricted by hospital visiting hours. Hospitals that maintain these restrictive policies cite concerns about negative impacts on security, infection control, privacy, and staff workload. But there are no data to support these concerns, and the experience of hospitals that have successfully changed policy and practice to welcome families demonstrates the potential positive impacts of less restrictive policies on patient care and outcomes.¹ For example, hospitalization can lead to reduced cognitive function in elderly patients. Family members would recognize the changes and could provide valuable information to hospital staff, potentially improving outcomes.²

In June 2014, the Institute for Patient- and Family-Centered Care (IPFCC) launched the campaign Better Together: Partnering with Families.³ The campaign is is grounded in patient- and family- centered care, an approach to care that supports partnerships among health care providers, patients, and families, and, among other core principles, advocates that patients define their “families” and how they will participate in care and decision-making.

Emphasizing the importance of family presence and participation to quality and safety, the Better Together campaign seeks to eliminate restrictive visiting policies and calls upon hospitals to include families as members of the care team and to welcome them 24 hours a day, 7 days a week, according to patient preference. As part of the campaign, IPFCC developed an extensive toolkit of resources that is available to hospitals and other organizations at no cost. The resources include sample policies; profiles of hospitals that have implemented family presence policies; educational materials for staff, patients, and families; and a template for hospital websites. This article, a follow-up to an article published in the January 2015 issue of JCOM,¹ discusses the growth of the Better Together initiative as well as the processes involved in implementing the initiative across a large health system.

Growth of the Initiative

Since its launch in 2014, the Better Together initiative has continued to expand in the United States and Canada. In Canada, under the leadership of the Canadian Foundation for Healthcare Improvement (CFHI), more than 50 organizations have made a commitment to the Better Together program and family presence.⁴ Utilizing and adapting IPFCC’s Toolkit, CFHI developed a change package of free resources for Canadian organizations.⁵ Some of the materials, including the Pocket Guide for Families (Manuel des Familles), were translated into French.⁶

With support from 2 funders in the United States, the United Hospital Fund and the New York State Health (NYSHealth) Foundation, through a subcontract with the New York Public Interest Research Group (NYPIRG), IPFCC has been able to focus on hospitals in New York City, including public hospitals, and, more broadly, acute care hospitals across New York State. Nearly 50 hospitals participated in these 2 separate but related projects.

Education and Support for New York City Hospitals

Supported by the United Hospital Fund, an 18-month project that focused specifically on New York City hospitals was completed in June 2017. The project began with a 1-day intensive training event with representatives of 21 hospitals. Eighteen of those hospitals were eligible to participate in follow-up consultation provided by IPFCC, and 14 participated in some kind of follow-up. NYC Health + Hospitals (H+H), the system of public hospitals in NYC, participated most fully in these activities.

The outcomes of the Better Together initiative in New York City are summarized in the report Sick, Scared, & Separated From Loved Ones,² which is based on a pre/post review of hospital visitation/family presence policies and website communications. According to the report, hospitals that participated in the IPFCC training and consultation program performed better, as a group, with respect to improved policy and website scores on post review than those that did not. Of the 10 hospitals whose scores improved during the review period, 8 had participated in the IPFCC training and 1 hospital was part of a hospital network that did so. (Six of these hospitals are part of the H+H public hospital system.) Those 9 hospitals saw an average increase in scores of 4.9 points (out of a possible 11). All of the website communication improvements were related to the designation or role of the family member/care partner, or the patient’s right to choose visitors and family members/care partners, fundamental elements of the Better Together initiative.²

A Learning Community for Hospitals in New York State

With support from the NYSHealth Foundation, IPFCC again collaborated with NYPIRG and New Yorkers for Patient & Family Empowerment on a 2-year initiative, completed in November 2019, that involved 26 hospitals: 15 from Northwell Health, New York State’s largest health system, and 11 hospitals from health systems throughout the state (Greater Hudson Valley Health System, now Garnet Health; Mohawk Valley Health System; Rochester Regional Health; and University of Vermont Health Network). An update of the report Sick, Scared, & Separated From Loved Onescompared pre/post reviews of policies and website communications regarding hospital visitation/family presence.⁷ Its findings confirm that hospitals that participated in the Better Together Learning Community improved both their policy and website scores to a greater degree than hospitals that did not participate and that a planned intervention can help facilitate change.

During the survey period, 28 out of 40 hospitals’ website navigability scores improved. Of those, hospitals that did not participate in the Better Together Learning Community saw an average increase in scores of 1.2 points, out of a possible 11, while the participating hospitals saw an average increase of 2.7 points, with the top 5 largest increases in scores belonging to hospitals that participated in the Better Together Learning Community.⁷

Northwell Health is a large integrated health care organization comprising more than 69,000 employees, 23 hospitals, and more than 750 medical practices, located geographically across New York State. Embracing patient- and family-centered care, Northwell is dedicated to improving the quality, experience, and safety of care for patients and their families. Welcoming and including patients, families, and care partners as members of the health care team has always been a core element of Northwell’s organizational goal of providing world-class patient care and experience.

Four years ago, the organization reorganized and formalized a system-wide Patient & Family Partnership Council (PFPC).⁸ Representatives on the PFPC include a Northwell patient experience leader and patient/family co-chair from local councils that have been established in nearly all 23 hospitals as well as service lines. Modeling partnership, the PFPC is grounded in listening to the “voice” of patients and families and promoting collaboration, with the goal of driving change across varied aspects and experiences of health care delivery.

Through the Office of Patient and Customer Experience (OPCE), a partnership with IPFCC and the Better Together Learning Community for Hospitals in New York State was initiated as a fundamental next step in Northwell’s journey to enhance system-wide family presence and participation. Results from Better Together’s Organizational Self-Assessment Tool and process identified opportunities to influence 3 distinct areas: policy/staff education, position descriptions/performance management, and website/signage. Over a 10-month period (September 2018 through June 2019), 15 Northwell hospitals implemened significant patient- and family-centered improvements through multifaceted shared work teams (SWT) that partnered around the common goal of supporting the patient and family experience (Figure). Northwell’s SWT structure allowed teams to meet individually on specific tasks, led by a dedicated staff member of the OPCE to ensure progress, support, and accountability. Six monthly coaching calls or report-out meetings were attended by participating teams, where feedback and recommendations shared by IPFCC were discussed in order to maintain momentum and results.

Policy/Staff Education

The policy/staff education SWT focused on appraising and updating existing policies to ensure alignment with key patient- and family-centered concepts and Better Together principles (Table 1). By establishing representation on the System Policy and Procedure Committee, OPCE enabled patients and families to have a voice at the decision-making table. OPCE leaders presented the ideology and scope of the transformation to this committee. After reviewing all system-wide policies, 4 were identified as key opportunities for revision. One overarching policy titled “Visitation Guidelines” was reviewed and updated to reflect Northwell’s mission of patient- and family-centered care, retiring the reference to “families” as “visitors” in definitions, incorporating language of inclusion and partnership, and citing other related policies. The policy was vetted through a multilayer process of review and stakeholder feedback and was ultimately approved at a system Performance Improvement Coordinating Group meeting under a new title, “Visitation: Presence and Participation of Families, Support System Designees and Visitor(s) in Care.”

Three additional related policies were also updated to reflect core principles of inclusion and partnership. These included system policies focused on discharge planning; identification of health care proxy, agent, support person and caregiver; and standards of behavior not conducive in a health care setting. As a result of this work, OPCE was invited to remain an active member of the System Policy and Procedure Committee, adding meaningful new perspectives to the clinical and administrative policy management process. Once policies were updated and approved, the SWT focused on educating leaders and teams. Using a diversified strategy, education was provided through various modes, including weekly system-wide internal communication channels, patient experience huddle messages, yearly mandatory topics training, and the incorporation of essential concepts in existing educational courses (classroom and e-learning modalities).

Position Descriptions/Performance Management

The position descriptions/performance management SWT focused its efforts on incorporating patient- and family-centered concepts and language into position descriptions and the performance appraisal process (Table 2). Due to the complex nature of this work, the process required collaboration from key subject matter experts in human resources, talent management, corporate compensation, and labor management. In 2019, Northwell began an initiative focused on streamlining and standardizing job titles, roles, and developmental pathways across the system. The overarching goal was to create system-wide consistency and standardization. The SWT was successful in advising the leaders overseeing this job architecture initiative on the importance of including language of patient- and family-centered care, like partnership and collaboration, and of highlighting the critical role of family members as part of the care team in subsequent documents.

Northwell has 6 behavioral expectations, standards to which all team members are held accountable: Patient/Customer Focus, Teamwork, Execution, Organizational Awareness, Enable Change, and Develop Self. As a result of the SWT’s work, Patient/Customer Focus was revised to include “families” as essential care partners, demonstrating Northwell’s ongoing commitment to honoring the role of families as members of the care team. It also ensures that all employees are aligned around this priority, as these expectations are utilized to support areas such as recognition and performance. Collaborating with talent management and organizational development, the SWT reviewed yearly performance management and new-hire evaluations. In doing so, they identified an opportunity to refresh the anchored qualitative rating scales to include behavioral demonstrations of patient- and family-centered care, collaboration, respect, and partnership with family members.

Website/Signage

Websites make an important first impression on patients and families looking for information to best prepare for a hospital experience. Therefore, the website/signage SWT worked to redesign hospital websites, enhance digital signage, and perform a baseline assessment of physical signage across facilities. Initial feedback on Northwell’s websites identified opportunities to include more patient- and family-centered, care-partner-infused language; improve navigation; and streamline click levels for easier access. Content for the websites was carefully crafted in collaboration with Northwell’s internal web team, utilizing IPFCC’s best practice standards as a framework and guide.

Next, a multidisciplinary website shared-governance team was established by the OPCE to ensure that key stakeholders were represented and had the opportunity to review and make recommendations for appropriate language and messaging about family presence and participation. This 13-person team was comprised of patient/family partners, patient-experience culture leaders, quality, compliance, human resources, policy, a chief nursing officer, a medical director, and representation from the Institute for Nursing. After careful review and consideration from Northwell’s family partners and teams, all participating hospital websites were enhanced as of June 2019 to include prominent 1-click access from homepages to information for “patients, families and visitors,” as well as “your care partners” information on the important role of families and care partners.

Along with refreshing websites, another step in Northwell’s work to strengthen messaging about family presence and participation was to partner and collaborate with the system’s digital web team as well as local facility councils to understand the capacity to adjust digital signage across facilities. Opportunities were found to make simple yet effective enhancements to the language and imagery of digital signage upon entry, creating a warmer and more welcoming first impression for patients and families. With patient and family partner feedback, the team designed digital signage with inclusive messaging and images that would circulate appropriately based on the facility. Signage specifically welcomes families and refers to them as members of patients’ care teams.

Northwell’s website/signage SWT also directed a 2-phase physical signage assessment to determine ongoing opportunities to alter signs in areas that particularly impact patients and families, such as emergency departments, main lobbies, cafeterias, surgical waiting areas, and intensive care units. Each hospital’s local PFPC did a “walk-about”⁹ to make enhancements to physical signage, such as removing paper and overcrowded signs, adjusting negative language, ensuring alignment with brand guidelines, and including language that welcomed families. As a result of the team’s efforts around signage, collaboration began with the health system’s signage committee to help standardize signage terminology to reflect family inclusiveness, and to implement the recommendation for a standardized signage shared-governance team to ensure accountability and a patient- and family-centered structure.

Sustainment

Since implementing Better Together, Northwell has been able to infuse a more patient- and family-centered emphasis into its overall patient experience message of “Every role, every person, every moment matters.” As a strategic tool aimed at encouraging leaders, clinicians, and staff to pause and reflect about the “heart” of their work, patient and family stories are now included at the beginning of meetings, forums, and team huddles. Elements of the initiative have been integrated in current Patient and Family Partnership sustainment plans at participating hospitals. Some highlights include continued integration of patient/family partners on committees and councils that impact areas such as way finding, signage, recruitment, new-hire orientation, and community outreach; focus on enhancing partner retention and development programs; and inclusion of patient- and family-centered care and Better Together principles in ongoing leadership meetings.

Factors Contributing to Success

Health care is a complex, regulated, and often bureaucratic world that can be very difficult for patients and families to navigate. The system’s partnership with the Better Together Learning Community for Hospitals in New York State enhanced its efforts to improve family presence and participation and created powerful synergy. The success of this partnership was based on a number of important factors:

A solid foundation of support, structure, and accountability. The OPCE initiated the IPFCC Better Together partnership and established a synergistic collaboration inclusive of leadership, frontline teams, multiple departments, and patient and family partners. As a major strategic component of Northwell’s mission to deliver high-quality, patient- and family-centered care, OPCE was instrumental in connecting key areas and stakeholders and mobilizing the recommendations coming from patients and families.

A visible commitment of leadership at all levels. Partnering with leadership across Northwell’s system required a delineated vision, clear purpose and ownership, and comprehensive implementation and sustainment strategies. The existing format of Northwell’s PFPC provided the structure and framework needed for engaged patient and family input; the OPCE motivated and organized key areas of involvement and led communication efforts across the organization. The IPFCC coaching calls provided the underlying guidance and accountability needed to sustain momentum. As leadership and frontline teams became aware of the vision, they understood the larger connection to the system’s purpose, which ultimately created a clear path for positive change.

Meaningful involvement and input of patient and family partners. Throughout this project, Northwell’s patient/family partners were involved through the PFPC and local councils. For example, patient/family partners attended every IPFCC coaching call; members had a central voice in every decision made within each SWT; and local PFPCs actively participated in physical signage “walk-abouts” across facilities, making key recommendations for improvement. This multifaceted, supportive collaboration created a rejuvenated and purposeful focus for all council members involved. Some of their reactions include, “…I am so happy to be able to help other families in crisis, so that they don’t have to be alone, like I was,” and “I feel how important the patient and family’s voice is … it’s truly a partnership between patients, families, and staff.”

Regular access to IPFCC as a best practice coach and expert resource. Throughout the 10-month process, IPFCC’s Better Together Learning Community for Hospitals in New York State provided ongoing learning interventions for members of the SWT; multiple and varied resources from the Better Together toolkit for adaptation; and opportunities to share and reinforce new, learned expertise with colleagues within the Northwell Health system and beyond through IPFCC’s free online learning community, PFCC.Connect.

Conclusion

Family presence and participation are important to the quality, experience, safety, and outcomes of care. IPFCC’s campaign, Better Together: Partnering with Families, encourages hospitals to change restrictive visiting policies and, instead, to welcome families and caregivers 24 hours a day.

Two projects within Better Together involving almost 50 acute care hospitals in New York State confirm that change in policy, practice, and communication is particularly effective when implemented with strong support from leadership. An intervention like the Better Together Learning Community, offering structured training, coaching, and resources, can facilitate the change process.

Corresponding author: IPFCC, Deborah L. Dokken, 6917 Arlington Rd., Ste. 309, Bethesda, MD 20814; [email protected].

Funding disclosures: None.

From the Institute for Patient- and Family-Centered Care, Bethesda, MD (Ms. Dokken and Ms. Johnson), and Northwell Health, New Hyde Park, NY (Dr. Barden, Ms. Tuomey, and Ms. Giammarinaro).

Abstract

Objective: To describe the growth of Better Together: Partnering with Families, a campaign launched in 2014 to eliminate restrictive hospital visiting policies and to put in place policies that recognize families as partners in care, and to discuss the processes involved in implementing the initiative in a large, integrated health system.

Methods: Descriptive report.

Results: In June 2014, the Institute for Patient- and Family-Centered Care (IPFCC) launched the Better Together campaign to emphasize the importance of family presence and participation to the quality, experience, safety, and outcomes of care. Since then, this initiative has expanded in both the United States and Canada. With support from 2 funders in the United States, special attention was focused on acute care hospitals across New York State. Nearly 50 hospitals participated in 2 separate but related projects. Fifteen of the hospitals are part of Northwell Health, New York State’s largest health system. Over a 10-month period, these hospitals made significant progress in changing policy, practice, and communication to support family presence.

Conclusion: The Better Together initiative was implemented across a health system with strong support from leadership and the involvement of patient and family advisors. An intervention offering structured training, coaching, and resources, like IPFCC’s Better Together initiative, can facilitate the change process.

Keywords: family presence; visiting policies; patient-centered care; family-centered care; patient experience.

The presence of families at the bedside of patients is often restricted by hospital visiting hours. Hospitals that maintain these restrictive policies cite concerns about negative impacts on security, infection control, privacy, and staff workload. But there are no data to support these concerns, and the experience of hospitals that have successfully changed policy and practice to welcome families demonstrates the potential positive impacts of less restrictive policies on patient care and outcomes.¹ For example, hospitalization can lead to reduced cognitive function in elderly patients. Family members would recognize the changes and could provide valuable information to hospital staff, potentially improving outcomes.²

In June 2014, the Institute for Patient- and Family-Centered Care (IPFCC) launched the campaign Better Together: Partnering with Families.³ The campaign is is grounded in patient- and family- centered care, an approach to care that supports partnerships among health care providers, patients, and families, and, among other core principles, advocates that patients define their “families” and how they will participate in care and decision-making.

Emphasizing the importance of family presence and participation to quality and safety, the Better Together campaign seeks to eliminate restrictive visiting policies and calls upon hospitals to include families as members of the care team and to welcome them 24 hours a day, 7 days a week, according to patient preference. As part of the campaign, IPFCC developed an extensive toolkit of resources that is available to hospitals and other organizations at no cost. The resources include sample policies; profiles of hospitals that have implemented family presence policies; educational materials for staff, patients, and families; and a template for hospital websites. This article, a follow-up to an article published in the January 2015 issue of JCOM,¹ discusses the growth of the Better Together initiative as well as the processes involved in implementing the initiative across a large health system.

Growth of the Initiative

Since its launch in 2014, the Better Together initiative has continued to expand in the United States and Canada. In Canada, under the leadership of the Canadian Foundation for Healthcare Improvement (CFHI), more than 50 organizations have made a commitment to the Better Together program and family presence.⁴ Utilizing and adapting IPFCC’s Toolkit, CFHI developed a change package of free resources for Canadian organizations.⁵ Some of the materials, including the Pocket Guide for Families (Manuel des Familles), were translated into French.⁶

With support from 2 funders in the United States, the United Hospital Fund and the New York State Health (NYSHealth) Foundation, through a subcontract with the New York Public Interest Research Group (NYPIRG), IPFCC has been able to focus on hospitals in New York City, including public hospitals, and, more broadly, acute care hospitals across New York State. Nearly 50 hospitals participated in these 2 separate but related projects.

Education and Support for New York City Hospitals

Supported by the United Hospital Fund, an 18-month project that focused specifically on New York City hospitals was completed in June 2017. The project began with a 1-day intensive training event with representatives of 21 hospitals. Eighteen of those hospitals were eligible to participate in follow-up consultation provided by IPFCC, and 14 participated in some kind of follow-up. NYC Health + Hospitals (H+H), the system of public hospitals in NYC, participated most fully in these activities.

The outcomes of the Better Together initiative in New York City are summarized in the report Sick, Scared, & Separated From Loved Ones,² which is based on a pre/post review of hospital visitation/family presence policies and website communications. According to the report, hospitals that participated in the IPFCC training and consultation program performed better, as a group, with respect to improved policy and website scores on post review than those that did not. Of the 10 hospitals whose scores improved during the review period, 8 had participated in the IPFCC training and 1 hospital was part of a hospital network that did so. (Six of these hospitals are part of the H+H public hospital system.) Those 9 hospitals saw an average increase in scores of 4.9 points (out of a possible 11). All of the website communication improvements were related to the designation or role of the family member/care partner, or the patient’s right to choose visitors and family members/care partners, fundamental elements of the Better Together initiative.²

A Learning Community for Hospitals in New York State

With support from the NYSHealth Foundation, IPFCC again collaborated with NYPIRG and New Yorkers for Patient & Family Empowerment on a 2-year initiative, completed in November 2019, that involved 26 hospitals: 15 from Northwell Health, New York State’s largest health system, and 11 hospitals from health systems throughout the state (Greater Hudson Valley Health System, now Garnet Health; Mohawk Valley Health System; Rochester Regional Health; and University of Vermont Health Network). An update of the report Sick, Scared, & Separated From Loved Onescompared pre/post reviews of policies and website communications regarding hospital visitation/family presence.⁷ Its findings confirm that hospitals that participated in the Better Together Learning Community improved both their policy and website scores to a greater degree than hospitals that did not participate and that a planned intervention can help facilitate change.

During the survey period, 28 out of 40 hospitals’ website navigability scores improved. Of those, hospitals that did not participate in the Better Together Learning Community saw an average increase in scores of 1.2 points, out of a possible 11, while the participating hospitals saw an average increase of 2.7 points, with the top 5 largest increases in scores belonging to hospitals that participated in the Better Together Learning Community.⁷

Northwell Health is a large integrated health care organization comprising more than 69,000 employees, 23 hospitals, and more than 750 medical practices, located geographically across New York State. Embracing patient- and family-centered care, Northwell is dedicated to improving the quality, experience, and safety of care for patients and their families. Welcoming and including patients, families, and care partners as members of the health care team has always been a core element of Northwell’s organizational goal of providing world-class patient care and experience.

Four years ago, the organization reorganized and formalized a system-wide Patient & Family Partnership Council (PFPC).⁸ Representatives on the PFPC include a Northwell patient experience leader and patient/family co-chair from local councils that have been established in nearly all 23 hospitals as well as service lines. Modeling partnership, the PFPC is grounded in listening to the “voice” of patients and families and promoting collaboration, with the goal of driving change across varied aspects and experiences of health care delivery.

Through the Office of Patient and Customer Experience (OPCE), a partnership with IPFCC and the Better Together Learning Community for Hospitals in New York State was initiated as a fundamental next step in Northwell’s journey to enhance system-wide family presence and participation. Results from Better Together’s Organizational Self-Assessment Tool and process identified opportunities to influence 3 distinct areas: policy/staff education, position descriptions/performance management, and website/signage. Over a 10-month period (September 2018 through June 2019), 15 Northwell hospitals implemened significant patient- and family-centered improvements through multifaceted shared work teams (SWT) that partnered around the common goal of supporting the patient and family experience (Figure). Northwell’s SWT structure allowed teams to meet individually on specific tasks, led by a dedicated staff member of the OPCE to ensure progress, support, and accountability. Six monthly coaching calls or report-out meetings were attended by participating teams, where feedback and recommendations shared by IPFCC were discussed in order to maintain momentum and results.

Policy/Staff Education

The policy/staff education SWT focused on appraising and updating existing policies to ensure alignment with key patient- and family-centered concepts and Better Together principles (Table 1). By establishing representation on the System Policy and Procedure Committee, OPCE enabled patients and families to have a voice at the decision-making table. OPCE leaders presented the ideology and scope of the transformation to this committee. After reviewing all system-wide policies, 4 were identified as key opportunities for revision. One overarching policy titled “Visitation Guidelines” was reviewed and updated to reflect Northwell’s mission of patient- and family-centered care, retiring the reference to “families” as “visitors” in definitions, incorporating language of inclusion and partnership, and citing other related policies. The policy was vetted through a multilayer process of review and stakeholder feedback and was ultimately approved at a system Performance Improvement Coordinating Group meeting under a new title, “Visitation: Presence and Participation of Families, Support System Designees and Visitor(s) in Care.”

Three additional related policies were also updated to reflect core principles of inclusion and partnership. These included system policies focused on discharge planning; identification of health care proxy, agent, support person and caregiver; and standards of behavior not conducive in a health care setting. As a result of this work, OPCE was invited to remain an active member of the System Policy and Procedure Committee, adding meaningful new perspectives to the clinical and administrative policy management process. Once policies were updated and approved, the SWT focused on educating leaders and teams. Using a diversified strategy, education was provided through various modes, including weekly system-wide internal communication channels, patient experience huddle messages, yearly mandatory topics training, and the incorporation of essential concepts in existing educational courses (classroom and e-learning modalities).

Position Descriptions/Performance Management

The position descriptions/performance management SWT focused its efforts on incorporating patient- and family-centered concepts and language into position descriptions and the performance appraisal process (Table 2). Due to the complex nature of this work, the process required collaboration from key subject matter experts in human resources, talent management, corporate compensation, and labor management. In 2019, Northwell began an initiative focused on streamlining and standardizing job titles, roles, and developmental pathways across the system. The overarching goal was to create system-wide consistency and standardization. The SWT was successful in advising the leaders overseeing this job architecture initiative on the importance of including language of patient- and family-centered care, like partnership and collaboration, and of highlighting the critical role of family members as part of the care team in subsequent documents.

Northwell has 6 behavioral expectations, standards to which all team members are held accountable: Patient/Customer Focus, Teamwork, Execution, Organizational Awareness, Enable Change, and Develop Self. As a result of the SWT’s work, Patient/Customer Focus was revised to include “families” as essential care partners, demonstrating Northwell’s ongoing commitment to honoring the role of families as members of the care team. It also ensures that all employees are aligned around this priority, as these expectations are utilized to support areas such as recognition and performance. Collaborating with talent management and organizational development, the SWT reviewed yearly performance management and new-hire evaluations. In doing so, they identified an opportunity to refresh the anchored qualitative rating scales to include behavioral demonstrations of patient- and family-centered care, collaboration, respect, and partnership with family members.

Website/Signage

Websites make an important first impression on patients and families looking for information to best prepare for a hospital experience. Therefore, the website/signage SWT worked to redesign hospital websites, enhance digital signage, and perform a baseline assessment of physical signage across facilities. Initial feedback on Northwell’s websites identified opportunities to include more patient- and family-centered, care-partner-infused language; improve navigation; and streamline click levels for easier access. Content for the websites was carefully crafted in collaboration with Northwell’s internal web team, utilizing IPFCC’s best practice standards as a framework and guide.

Next, a multidisciplinary website shared-governance team was established by the OPCE to ensure that key stakeholders were represented and had the opportunity to review and make recommendations for appropriate language and messaging about family presence and participation. This 13-person team was comprised of patient/family partners, patient-experience culture leaders, quality, compliance, human resources, policy, a chief nursing officer, a medical director, and representation from the Institute for Nursing. After careful review and consideration from Northwell’s family partners and teams, all participating hospital websites were enhanced as of June 2019 to include prominent 1-click access from homepages to information for “patients, families and visitors,” as well as “your care partners” information on the important role of families and care partners.

Along with refreshing websites, another step in Northwell’s work to strengthen messaging about family presence and participation was to partner and collaborate with the system’s digital web team as well as local facility councils to understand the capacity to adjust digital signage across facilities. Opportunities were found to make simple yet effective enhancements to the language and imagery of digital signage upon entry, creating a warmer and more welcoming first impression for patients and families. With patient and family partner feedback, the team designed digital signage with inclusive messaging and images that would circulate appropriately based on the facility. Signage specifically welcomes families and refers to them as members of patients’ care teams.

Northwell’s website/signage SWT also directed a 2-phase physical signage assessment to determine ongoing opportunities to alter signs in areas that particularly impact patients and families, such as emergency departments, main lobbies, cafeterias, surgical waiting areas, and intensive care units. Each hospital’s local PFPC did a “walk-about”⁹ to make enhancements to physical signage, such as removing paper and overcrowded signs, adjusting negative language, ensuring alignment with brand guidelines, and including language that welcomed families. As a result of the team’s efforts around signage, collaboration began with the health system’s signage committee to help standardize signage terminology to reflect family inclusiveness, and to implement the recommendation for a standardized signage shared-governance team to ensure accountability and a patient- and family-centered structure.

Sustainment

Since implementing Better Together, Northwell has been able to infuse a more patient- and family-centered emphasis into its overall patient experience message of “Every role, every person, every moment matters.” As a strategic tool aimed at encouraging leaders, clinicians, and staff to pause and reflect about the “heart” of their work, patient and family stories are now included at the beginning of meetings, forums, and team huddles. Elements of the initiative have been integrated in current Patient and Family Partnership sustainment plans at participating hospitals. Some highlights include continued integration of patient/family partners on committees and councils that impact areas such as way finding, signage, recruitment, new-hire orientation, and community outreach; focus on enhancing partner retention and development programs; and inclusion of patient- and family-centered care and Better Together principles in ongoing leadership meetings.

Factors Contributing to Success

Health care is a complex, regulated, and often bureaucratic world that can be very difficult for patients and families to navigate. The system’s partnership with the Better Together Learning Community for Hospitals in New York State enhanced its efforts to improve family presence and participation and created powerful synergy. The success of this partnership was based on a number of important factors:

A solid foundation of support, structure, and accountability. The OPCE initiated the IPFCC Better Together partnership and established a synergistic collaboration inclusive of leadership, frontline teams, multiple departments, and patient and family partners. As a major strategic component of Northwell’s mission to deliver high-quality, patient- and family-centered care, OPCE was instrumental in connecting key areas and stakeholders and mobilizing the recommendations coming from patients and families.

A visible commitment of leadership at all levels. Partnering with leadership across Northwell’s system required a delineated vision, clear purpose and ownership, and comprehensive implementation and sustainment strategies. The existing format of Northwell’s PFPC provided the structure and framework needed for engaged patient and family input; the OPCE motivated and organized key areas of involvement and led communication efforts across the organization. The IPFCC coaching calls provided the underlying guidance and accountability needed to sustain momentum. As leadership and frontline teams became aware of the vision, they understood the larger connection to the system’s purpose, which ultimately created a clear path for positive change.

Meaningful involvement and input of patient and family partners. Throughout this project, Northwell’s patient/family partners were involved through the PFPC and local councils. For example, patient/family partners attended every IPFCC coaching call; members had a central voice in every decision made within each SWT; and local PFPCs actively participated in physical signage “walk-abouts” across facilities, making key recommendations for improvement. This multifaceted, supportive collaboration created a rejuvenated and purposeful focus for all council members involved. Some of their reactions include, “…I am so happy to be able to help other families in crisis, so that they don’t have to be alone, like I was,” and “I feel how important the patient and family’s voice is … it’s truly a partnership between patients, families, and staff.”

Regular access to IPFCC as a best practice coach and expert resource. Throughout the 10-month process, IPFCC’s Better Together Learning Community for Hospitals in New York State provided ongoing learning interventions for members of the SWT; multiple and varied resources from the Better Together toolkit for adaptation; and opportunities to share and reinforce new, learned expertise with colleagues within the Northwell Health system and beyond through IPFCC’s free online learning community, PFCC.Connect.

Conclusion

Family presence and participation are important to the quality, experience, safety, and outcomes of care. IPFCC’s campaign, Better Together: Partnering with Families, encourages hospitals to change restrictive visiting policies and, instead, to welcome families and caregivers 24 hours a day.

Two projects within Better Together involving almost 50 acute care hospitals in New York State confirm that change in policy, practice, and communication is particularly effective when implemented with strong support from leadership. An intervention like the Better Together Learning Community, offering structured training, coaching, and resources, can facilitate the change process.

Corresponding author: IPFCC, Deborah L. Dokken, 6917 Arlington Rd., Ste. 309, Bethesda, MD 20814; [email protected].

Funding disclosures: None.

From the Institute for Patient- and Family-Centered Care, Bethesda, MD (Ms. Dokken and Ms. Johnson), and Northwell Health, New Hyde Park, NY (Dr. Barden, Ms. Tuomey, and Ms. Giammarinaro).

Abstract

Objective: To describe the growth of Better Together: Partnering with Families, a campaign launched in 2014 to eliminate restrictive hospital visiting policies and to put in place policies that recognize families as partners in care, and to discuss the processes involved in implementing the initiative in a large, integrated health system.

Methods: Descriptive report.

Results: In June 2014, the Institute for Patient- and Family-Centered Care (IPFCC) launched the Better Together campaign to emphasize the importance of family presence and participation to the quality, experience, safety, and outcomes of care. Since then, this initiative has expanded in both the United States and Canada. With support from 2 funders in the United States, special attention was focused on acute care hospitals across New York State. Nearly 50 hospitals participated in 2 separate but related projects. Fifteen of the hospitals are part of Northwell Health, New York State’s largest health system. Over a 10-month period, these hospitals made significant progress in changing policy, practice, and communication to support family presence.

Conclusion: The Better Together initiative was implemented across a health system with strong support from leadership and the involvement of patient and family advisors. An intervention offering structured training, coaching, and resources, like IPFCC’s Better Together initiative, can facilitate the change process.

Keywords: family presence; visiting policies; patient-centered care; family-centered care; patient experience.

The presence of families at the bedside of patients is often restricted by hospital visiting hours. Hospitals that maintain these restrictive policies cite concerns about negative impacts on security, infection control, privacy, and staff workload. But there are no data to support these concerns, and the experience of hospitals that have successfully changed policy and practice to welcome families demonstrates the potential positive impacts of less restrictive policies on patient care and outcomes.¹ For example, hospitalization can lead to reduced cognitive function in elderly patients. Family members would recognize the changes and could provide valuable information to hospital staff, potentially improving outcomes.²

In June 2014, the Institute for Patient- and Family-Centered Care (IPFCC) launched the campaign Better Together: Partnering with Families.³ The campaign is is grounded in patient- and family- centered care, an approach to care that supports partnerships among health care providers, patients, and families, and, among other core principles, advocates that patients define their “families” and how they will participate in care and decision-making.

Emphasizing the importance of family presence and participation to quality and safety, the Better Together campaign seeks to eliminate restrictive visiting policies and calls upon hospitals to include families as members of the care team and to welcome them 24 hours a day, 7 days a week, according to patient preference. As part of the campaign, IPFCC developed an extensive toolkit of resources that is available to hospitals and other organizations at no cost. The resources include sample policies; profiles of hospitals that have implemented family presence policies; educational materials for staff, patients, and families; and a template for hospital websites. This article, a follow-up to an article published in the January 2015 issue of JCOM,¹ discusses the growth of the Better Together initiative as well as the processes involved in implementing the initiative across a large health system.

Growth of the Initiative

Since its launch in 2014, the Better Together initiative has continued to expand in the United States and Canada. In Canada, under the leadership of the Canadian Foundation for Healthcare Improvement (CFHI), more than 50 organizations have made a commitment to the Better Together program and family presence.⁴ Utilizing and adapting IPFCC’s Toolkit, CFHI developed a change package of free resources for Canadian organizations.⁵ Some of the materials, including the Pocket Guide for Families (Manuel des Familles), were translated into French.⁶

With support from 2 funders in the United States, the United Hospital Fund and the New York State Health (NYSHealth) Foundation, through a subcontract with the New York Public Interest Research Group (NYPIRG), IPFCC has been able to focus on hospitals in New York City, including public hospitals, and, more broadly, acute care hospitals across New York State. Nearly 50 hospitals participated in these 2 separate but related projects.

Education and Support for New York City Hospitals

Supported by the United Hospital Fund, an 18-month project that focused specifically on New York City hospitals was completed in June 2017. The project began with a 1-day intensive training event with representatives of 21 hospitals. Eighteen of those hospitals were eligible to participate in follow-up consultation provided by IPFCC, and 14 participated in some kind of follow-up. NYC Health + Hospitals (H+H), the system of public hospitals in NYC, participated most fully in these activities.

The outcomes of the Better Together initiative in New York City are summarized in the report Sick, Scared, & Separated From Loved Ones,² which is based on a pre/post review of hospital visitation/family presence policies and website communications. According to the report, hospitals that participated in the IPFCC training and consultation program performed better, as a group, with respect to improved policy and website scores on post review than those that did not. Of the 10 hospitals whose scores improved during the review period, 8 had participated in the IPFCC training and 1 hospital was part of a hospital network that did so. (Six of these hospitals are part of the H+H public hospital system.) Those 9 hospitals saw an average increase in scores of 4.9 points (out of a possible 11). All of the website communication improvements were related to the designation or role of the family member/care partner, or the patient’s right to choose visitors and family members/care partners, fundamental elements of the Better Together initiative.²

A Learning Community for Hospitals in New York State

With support from the NYSHealth Foundation, IPFCC again collaborated with NYPIRG and New Yorkers for Patient & Family Empowerment on a 2-year initiative, completed in November 2019, that involved 26 hospitals: 15 from Northwell Health, New York State’s largest health system, and 11 hospitals from health systems throughout the state (Greater Hudson Valley Health System, now Garnet Health; Mohawk Valley Health System; Rochester Regional Health; and University of Vermont Health Network). An update of the report Sick, Scared, & Separated From Loved Onescompared pre/post reviews of policies and website communications regarding hospital visitation/family presence.⁷ Its findings confirm that hospitals that participated in the Better Together Learning Community improved both their policy and website scores to a greater degree than hospitals that did not participate and that a planned intervention can help facilitate change.

During the survey period, 28 out of 40 hospitals’ website navigability scores improved. Of those, hospitals that did not participate in the Better Together Learning Community saw an average increase in scores of 1.2 points, out of a possible 11, while the participating hospitals saw an average increase of 2.7 points, with the top 5 largest increases in scores belonging to hospitals that participated in the Better Together Learning Community.⁷

Northwell Health is a large integrated health care organization comprising more than 69,000 employees, 23 hospitals, and more than 750 medical practices, located geographically across New York State. Embracing patient- and family-centered care, Northwell is dedicated to improving the quality, experience, and safety of care for patients and their families. Welcoming and including patients, families, and care partners as members of the health care team has always been a core element of Northwell’s organizational goal of providing world-class patient care and experience.

Four years ago, the organization reorganized and formalized a system-wide Patient & Family Partnership Council (PFPC).⁸ Representatives on the PFPC include a Northwell patient experience leader and patient/family co-chair from local councils that have been established in nearly all 23 hospitals as well as service lines. Modeling partnership, the PFPC is grounded in listening to the “voice” of patients and families and promoting collaboration, with the goal of driving change across varied aspects and experiences of health care delivery.

Through the Office of Patient and Customer Experience (OPCE), a partnership with IPFCC and the Better Together Learning Community for Hospitals in New York State was initiated as a fundamental next step in Northwell’s journey to enhance system-wide family presence and participation. Results from Better Together’s Organizational Self-Assessment Tool and process identified opportunities to influence 3 distinct areas: policy/staff education, position descriptions/performance management, and website/signage. Over a 10-month period (September 2018 through June 2019), 15 Northwell hospitals implemened significant patient- and family-centered improvements through multifaceted shared work teams (SWT) that partnered around the common goal of supporting the patient and family experience (Figure). Northwell’s SWT structure allowed teams to meet individually on specific tasks, led by a dedicated staff member of the OPCE to ensure progress, support, and accountability. Six monthly coaching calls or report-out meetings were attended by participating teams, where feedback and recommendations shared by IPFCC were discussed in order to maintain momentum and results.

Policy/Staff Education

The policy/staff education SWT focused on appraising and updating existing policies to ensure alignment with key patient- and family-centered concepts and Better Together principles (Table 1). By establishing representation on the System Policy and Procedure Committee, OPCE enabled patients and families to have a voice at the decision-making table. OPCE leaders presented the ideology and scope of the transformation to this committee. After reviewing all system-wide policies, 4 were identified as key opportunities for revision. One overarching policy titled “Visitation Guidelines” was reviewed and updated to reflect Northwell’s mission of patient- and family-centered care, retiring the reference to “families” as “visitors” in definitions, incorporating language of inclusion and partnership, and citing other related policies. The policy was vetted through a multilayer process of review and stakeholder feedback and was ultimately approved at a system Performance Improvement Coordinating Group meeting under a new title, “Visitation: Presence and Participation of Families, Support System Designees and Visitor(s) in Care.”

Three additional related policies were also updated to reflect core principles of inclusion and partnership. These included system policies focused on discharge planning; identification of health care proxy, agent, support person and caregiver; and standards of behavior not conducive in a health care setting. As a result of this work, OPCE was invited to remain an active member of the System Policy and Procedure Committee, adding meaningful new perspectives to the clinical and administrative policy management process. Once policies were updated and approved, the SWT focused on educating leaders and teams. Using a diversified strategy, education was provided through various modes, including weekly system-wide internal communication channels, patient experience huddle messages, yearly mandatory topics training, and the incorporation of essential concepts in existing educational courses (classroom and e-learning modalities).

Position Descriptions/Performance Management

The position descriptions/performance management SWT focused its efforts on incorporating patient- and family-centered concepts and language into position descriptions and the performance appraisal process (Table 2). Due to the complex nature of this work, the process required collaboration from key subject matter experts in human resources, talent management, corporate compensation, and labor management. In 2019, Northwell began an initiative focused on streamlining and standardizing job titles, roles, and developmental pathways across the system. The overarching goal was to create system-wide consistency and standardization. The SWT was successful in advising the leaders overseeing this job architecture initiative on the importance of including language of patient- and family-centered care, like partnership and collaboration, and of highlighting the critical role of family members as part of the care team in subsequent documents.

Northwell has 6 behavioral expectations, standards to which all team members are held accountable: Patient/Customer Focus, Teamwork, Execution, Organizational Awareness, Enable Change, and Develop Self. As a result of the SWT’s work, Patient/Customer Focus was revised to include “families” as essential care partners, demonstrating Northwell’s ongoing commitment to honoring the role of families as members of the care team. It also ensures that all employees are aligned around this priority, as these expectations are utilized to support areas such as recognition and performance. Collaborating with talent management and organizational development, the SWT reviewed yearly performance management and new-hire evaluations. In doing so, they identified an opportunity to refresh the anchored qualitative rating scales to include behavioral demonstrations of patient- and family-centered care, collaboration, respect, and partnership with family members.

Website/Signage

Websites make an important first impression on patients and families looking for information to best prepare for a hospital experience. Therefore, the website/signage SWT worked to redesign hospital websites, enhance digital signage, and perform a baseline assessment of physical signage across facilities. Initial feedback on Northwell’s websites identified opportunities to include more patient- and family-centered, care-partner-infused language; improve navigation; and streamline click levels for easier access. Content for the websites was carefully crafted in collaboration with Northwell’s internal web team, utilizing IPFCC’s best practice standards as a framework and guide.

Next, a multidisciplinary website shared-governance team was established by the OPCE to ensure that key stakeholders were represented and had the opportunity to review and make recommendations for appropriate language and messaging about family presence and participation. This 13-person team was comprised of patient/family partners, patient-experience culture leaders, quality, compliance, human resources, policy, a chief nursing officer, a medical director, and representation from the Institute for Nursing. After careful review and consideration from Northwell’s family partners and teams, all participating hospital websites were enhanced as of June 2019 to include prominent 1-click access from homepages to information for “patients, families and visitors,” as well as “your care partners” information on the important role of families and care partners.

Along with refreshing websites, another step in Northwell’s work to strengthen messaging about family presence and participation was to partner and collaborate with the system’s digital web team as well as local facility councils to understand the capacity to adjust digital signage across facilities. Opportunities were found to make simple yet effective enhancements to the language and imagery of digital signage upon entry, creating a warmer and more welcoming first impression for patients and families. With patient and family partner feedback, the team designed digital signage with inclusive messaging and images that would circulate appropriately based on the facility. Signage specifically welcomes families and refers to them as members of patients’ care teams.

Northwell’s website/signage SWT also directed a 2-phase physical signage assessment to determine ongoing opportunities to alter signs in areas that particularly impact patients and families, such as emergency departments, main lobbies, cafeterias, surgical waiting areas, and intensive care units. Each hospital’s local PFPC did a “walk-about”⁹ to make enhancements to physical signage, such as removing paper and overcrowded signs, adjusting negative language, ensuring alignment with brand guidelines, and including language that welcomed families. As a result of the team’s efforts around signage, collaboration began with the health system’s signage committee to help standardize signage terminology to reflect family inclusiveness, and to implement the recommendation for a standardized signage shared-governance team to ensure accountability and a patient- and family-centered structure.

Sustainment

Since implementing Better Together, Northwell has been able to infuse a more patient- and family-centered emphasis into its overall patient experience message of “Every role, every person, every moment matters.” As a strategic tool aimed at encouraging leaders, clinicians, and staff to pause and reflect about the “heart” of their work, patient and family stories are now included at the beginning of meetings, forums, and team huddles. Elements of the initiative have been integrated in current Patient and Family Partnership sustainment plans at participating hospitals. Some highlights include continued integration of patient/family partners on committees and councils that impact areas such as way finding, signage, recruitment, new-hire orientation, and community outreach; focus on enhancing partner retention and development programs; and inclusion of patient- and family-centered care and Better Together principles in ongoing leadership meetings.

Factors Contributing to Success

Health care is a complex, regulated, and often bureaucratic world that can be very difficult for patients and families to navigate. The system’s partnership with the Better Together Learning Community for Hospitals in New York State enhanced its efforts to improve family presence and participation and created powerful synergy. The success of this partnership was based on a number of important factors:

A solid foundation of support, structure, and accountability. The OPCE initiated the IPFCC Better Together partnership and established a synergistic collaboration inclusive of leadership, frontline teams, multiple departments, and patient and family partners. As a major strategic component of Northwell’s mission to deliver high-quality, patient- and family-centered care, OPCE was instrumental in connecting key areas and stakeholders and mobilizing the recommendations coming from patients and families.

A visible commitment of leadership at all levels. Partnering with leadership across Northwell’s system required a delineated vision, clear purpose and ownership, and comprehensive implementation and sustainment strategies. The existing format of Northwell’s PFPC provided the structure and framework needed for engaged patient and family input; the OPCE motivated and organized key areas of involvement and led communication efforts across the organization. The IPFCC coaching calls provided the underlying guidance and accountability needed to sustain momentum. As leadership and frontline teams became aware of the vision, they understood the larger connection to the system’s purpose, which ultimately created a clear path for positive change.

Meaningful involvement and input of patient and family partners. Throughout this project, Northwell’s patient/family partners were involved through the PFPC and local councils. For example, patient/family partners attended every IPFCC coaching call; members had a central voice in every decision made within each SWT; and local PFPCs actively participated in physical signage “walk-abouts” across facilities, making key recommendations for improvement. This multifaceted, supportive collaboration created a rejuvenated and purposeful focus for all council members involved. Some of their reactions include, “…I am so happy to be able to help other families in crisis, so that they don’t have to be alone, like I was,” and “I feel how important the patient and family’s voice is … it’s truly a partnership between patients, families, and staff.”

Regular access to IPFCC as a best practice coach and expert resource. Throughout the 10-month process, IPFCC’s Better Together Learning Community for Hospitals in New York State provided ongoing learning interventions for members of the SWT; multiple and varied resources from the Better Together toolkit for adaptation; and opportunities to share and reinforce new, learned expertise with colleagues within the Northwell Health system and beyond through IPFCC’s free online learning community, PFCC.Connect.

Conclusion

Family presence and participation are important to the quality, experience, safety, and outcomes of care. IPFCC’s campaign, Better Together: Partnering with Families, encourages hospitals to change restrictive visiting policies and, instead, to welcome families and caregivers 24 hours a day.

Two projects within Better Together involving almost 50 acute care hospitals in New York State confirm that change in policy, practice, and communication is particularly effective when implemented with strong support from leadership. An intervention like the Better Together Learning Community, offering structured training, coaching, and resources, can facilitate the change process.

Corresponding author: IPFCC, Deborah L. Dokken, 6917 Arlington Rd., Ste. 309, Bethesda, MD 20814; [email protected].

Funding disclosures: None.

Safe, timely, and efficient diagnosis is fundamental for high-quality, effective healthcare. Why is diagnosis so important? First, it informs the two other main areas of medical decision-making: treatment and prognosis. These are the means by which physicians can actually change health outcomes for patients, as well as ensure that patients and their families have a realistic and accurate understanding of what the future holds with respect to their health. Second, patients and families tend to feel a sense of closure from having a name and an explanation for symptoms, even in the absence of specific treatment. Proper labeling allows patients and families to connect with others with the same diagnosis, who are best positioned to offer empathy by virtue of their similar experiences.

Despite the fundamental role of diagnosis, diagnostic error is pervasive in medicine, with unacceptable levels of resultant harm.¹ In 2015, the Institute of Medicine published a landmark report, “Improving Diagnosis in Health Care,” bringing the problem to the forefront of the minds of healthcare professionals and the general public alike. According to the report, “improving the diagnostic process…represents a moral, professional, and public health imperative.”¹ We must do more than avoid diagnostic error, however—we must aim to achieve diagnostic excellence. Not getting it wrong is not enough.

There are real challenges to achieving diagnostic safety, let alone excellence. The “churn” of modern hospital medicine does not reward deep diagnostic thought, nor does it often encourage reflection or collaboration, important components of being able to achieve diagnostic excellence.² Furthermore, despite their years of training, physicians often have difficulty applying probabilistic reasoning and appropriately incorporating diagnostic information in the best evidence-based manner.^3,4 In addition, there are no validated measures of diagnostic performance in practice. It is telling that many hospitalists, despite a professed interest in complex diagnosis, would rather be assigned to care for a patient with cellulitis than a patient with a complicated differential diagnosis.

Given these challenges, how can the modern healthcare ecosystem be changed to achieve diagnostic excellence? In this month’s issue of Journal of Hospital Medicine, Singer and colleagues describe a pilot project of a proposed solution to the problem.⁵ Aptly named, the Socrates Project is an intervention that makes available a team of “diagnosticians” that can be consulted for assistance with challenging diagnostic cases. The physicians on the team volunteer their time, allowing for deep diagnostic evaluation that is not limited by one’s daily workload, thus overcoming one of the major hurdles to achieving diagnostic excellence. The described program also focuses on harnessing the power of teamwork, which is especially relevant given recent descriptions of the effectiveness of collective intelligence in improving diagnostic performance.⁶ Importantly, the authors recognize that their intervention will not achieve a diagnosis in every case for which they are consulted; rather, they hope that their thorough evaluation will uncover additional potential diagnostic avenues for the referring team to pursue, with a goal to “improve patient care by providing…ideas to reduce—or at least manage—diagnostic uncertainty.”

Programs of this nature are exciting for hospitalists. Hospital medicine is, perhaps, a place in modern medicine where diagnostic excellence has a natural home. Patients admitted to the hospital are acutely (and often severely) ill, and hospitalists are tasked with rapidly identifying the cause of their illness in order to initiate appropriate treatment and accurately inform prognosis. Hospitalists, as generalists, take a broad approach to challenging cases, and they tend to practice in well-resourced environments with nearly every diagnostic modality at their disposal. Many hospitalists would envy participating in a program such as the Socrates Project.

While Singer et al.’s innovation—and the institutional support thereof—should be lauded, some discussion must be had about how to assess the effectiveness of such a program. The authors acknowledge the need for evaluation of both the diagnostic process and the outcomes that process achieves. Measuring diagnostic performance is challenging, however, and while there is substantial progress being made in this area, recent efforts tend to focus on identifying diagnostic errors rather than measuring diagnostic excellence. Moreover, even if a program does improve diagnostic performance, how should we evaluate for unintended consequences of its implementation? In the age of high-value care, how can we ensure that efforts to do a better job of spotting proverbial zebras do not come at the cost of harming too many horses?⁷

Hospitalists are well primed to answer this question. The juxtaposition of Singer et al.’s article with the Journal of Hospital Medicine’s long-running series on Choosing Wisely®: Things We Do for No Reason™ provides a natural synergy to begin crafting a framework to evaluate unintended consequences of a program in diagnostic excellence. More diagnosis is not the goal; more appropriate diagnosis is what is needed. A clinical program aimed at achieving diagnostic excellence should not employ low-value, wasteful strategies that do not add substantively to the diagnostic process but should instead seek to improve the overall efficiency of even complicated diagnostic odysseys. Avoiding waste throughout will allow for allocation of diagnostic resources where they are needed. In turn, hospitalists can do a better job of correctly identifying both horses and zebras for what they are. While a given hospitalization for a diagnostically complex patient may be relatively expensive, better diagnosis during an index hospitalization is likely to lead to decreased downstream costs, such as those related to readmissions and further testing, as well as better health outcomes.

The Socrates Project, along with similar programs at other institutions, are exciting innovations. These programs are not only likely to be good for patients but are also good for hospitalists. The field of hospital medicine should leverage its collective expertise in clinical medicine, systems of care, and high-value care to become a home for diagnostic excellence.

Safe, timely, and efficient diagnosis is fundamental for high-quality, effective healthcare. Why is diagnosis so important? First, it informs the two other main areas of medical decision-making: treatment and prognosis. These are the means by which physicians can actually change health outcomes for patients, as well as ensure that patients and their families have a realistic and accurate understanding of what the future holds with respect to their health. Second, patients and families tend to feel a sense of closure from having a name and an explanation for symptoms, even in the absence of specific treatment. Proper labeling allows patients and families to connect with others with the same diagnosis, who are best positioned to offer empathy by virtue of their similar experiences.

Despite the fundamental role of diagnosis, diagnostic error is pervasive in medicine, with unacceptable levels of resultant harm.¹ In 2015, the Institute of Medicine published a landmark report, “Improving Diagnosis in Health Care,” bringing the problem to the forefront of the minds of healthcare professionals and the general public alike. According to the report, “improving the diagnostic process…represents a moral, professional, and public health imperative.”¹ We must do more than avoid diagnostic error, however—we must aim to achieve diagnostic excellence. Not getting it wrong is not enough.

There are real challenges to achieving diagnostic safety, let alone excellence. The “churn” of modern hospital medicine does not reward deep diagnostic thought, nor does it often encourage reflection or collaboration, important components of being able to achieve diagnostic excellence.² Furthermore, despite their years of training, physicians often have difficulty applying probabilistic reasoning and appropriately incorporating diagnostic information in the best evidence-based manner.^3,4 In addition, there are no validated measures of diagnostic performance in practice. It is telling that many hospitalists, despite a professed interest in complex diagnosis, would rather be assigned to care for a patient with cellulitis than a patient with a complicated differential diagnosis.

Given these challenges, how can the modern healthcare ecosystem be changed to achieve diagnostic excellence? In this month’s issue of Journal of Hospital Medicine, Singer and colleagues describe a pilot project of a proposed solution to the problem.⁵ Aptly named, the Socrates Project is an intervention that makes available a team of “diagnosticians” that can be consulted for assistance with challenging diagnostic cases. The physicians on the team volunteer their time, allowing for deep diagnostic evaluation that is not limited by one’s daily workload, thus overcoming one of the major hurdles to achieving diagnostic excellence. The described program also focuses on harnessing the power of teamwork, which is especially relevant given recent descriptions of the effectiveness of collective intelligence in improving diagnostic performance.⁶ Importantly, the authors recognize that their intervention will not achieve a diagnosis in every case for which they are consulted; rather, they hope that their thorough evaluation will uncover additional potential diagnostic avenues for the referring team to pursue, with a goal to “improve patient care by providing…ideas to reduce—or at least manage—diagnostic uncertainty.”

Programs of this nature are exciting for hospitalists. Hospital medicine is, perhaps, a place in modern medicine where diagnostic excellence has a natural home. Patients admitted to the hospital are acutely (and often severely) ill, and hospitalists are tasked with rapidly identifying the cause of their illness in order to initiate appropriate treatment and accurately inform prognosis. Hospitalists, as generalists, take a broad approach to challenging cases, and they tend to practice in well-resourced environments with nearly every diagnostic modality at their disposal. Many hospitalists would envy participating in a program such as the Socrates Project.

While Singer et al.’s innovation—and the institutional support thereof—should be lauded, some discussion must be had about how to assess the effectiveness of such a program. The authors acknowledge the need for evaluation of both the diagnostic process and the outcomes that process achieves. Measuring diagnostic performance is challenging, however, and while there is substantial progress being made in this area, recent efforts tend to focus on identifying diagnostic errors rather than measuring diagnostic excellence. Moreover, even if a program does improve diagnostic performance, how should we evaluate for unintended consequences of its implementation? In the age of high-value care, how can we ensure that efforts to do a better job of spotting proverbial zebras do not come at the cost of harming too many horses?⁷

Hospitalists are well primed to answer this question. The juxtaposition of Singer et al.’s article with the Journal of Hospital Medicine’s long-running series on Choosing Wisely®: Things We Do for No Reason™ provides a natural synergy to begin crafting a framework to evaluate unintended consequences of a program in diagnostic excellence. More diagnosis is not the goal; more appropriate diagnosis is what is needed. A clinical program aimed at achieving diagnostic excellence should not employ low-value, wasteful strategies that do not add substantively to the diagnostic process but should instead seek to improve the overall efficiency of even complicated diagnostic odysseys. Avoiding waste throughout will allow for allocation of diagnostic resources where they are needed. In turn, hospitalists can do a better job of correctly identifying both horses and zebras for what they are. While a given hospitalization for a diagnostically complex patient may be relatively expensive, better diagnosis during an index hospitalization is likely to lead to decreased downstream costs, such as those related to readmissions and further testing, as well as better health outcomes.

The Socrates Project, along with similar programs at other institutions, are exciting innovations. These programs are not only likely to be good for patients but are also good for hospitalists. The field of hospital medicine should leverage its collective expertise in clinical medicine, systems of care, and high-value care to become a home for diagnostic excellence.

Safe, timely, and efficient diagnosis is fundamental for high-quality, effective healthcare. Why is diagnosis so important? First, it informs the two other main areas of medical decision-making: treatment and prognosis. These are the means by which physicians can actually change health outcomes for patients, as well as ensure that patients and their families have a realistic and accurate understanding of what the future holds with respect to their health. Second, patients and families tend to feel a sense of closure from having a name and an explanation for symptoms, even in the absence of specific treatment. Proper labeling allows patients and families to connect with others with the same diagnosis, who are best positioned to offer empathy by virtue of their similar experiences.

Despite the fundamental role of diagnosis, diagnostic error is pervasive in medicine, with unacceptable levels of resultant harm.¹ In 2015, the Institute of Medicine published a landmark report, “Improving Diagnosis in Health Care,” bringing the problem to the forefront of the minds of healthcare professionals and the general public alike. According to the report, “improving the diagnostic process…represents a moral, professional, and public health imperative.”¹ We must do more than avoid diagnostic error, however—we must aim to achieve diagnostic excellence. Not getting it wrong is not enough.

There are real challenges to achieving diagnostic safety, let alone excellence. The “churn” of modern hospital medicine does not reward deep diagnostic thought, nor does it often encourage reflection or collaboration, important components of being able to achieve diagnostic excellence.² Furthermore, despite their years of training, physicians often have difficulty applying probabilistic reasoning and appropriately incorporating diagnostic information in the best evidence-based manner.^3,4 In addition, there are no validated measures of diagnostic performance in practice. It is telling that many hospitalists, despite a professed interest in complex diagnosis, would rather be assigned to care for a patient with cellulitis than a patient with a complicated differential diagnosis.

Given these challenges, how can the modern healthcare ecosystem be changed to achieve diagnostic excellence? In this month’s issue of Journal of Hospital Medicine, Singer and colleagues describe a pilot project of a proposed solution to the problem.⁵ Aptly named, the Socrates Project is an intervention that makes available a team of “diagnosticians” that can be consulted for assistance with challenging diagnostic cases. The physicians on the team volunteer their time, allowing for deep diagnostic evaluation that is not limited by one’s daily workload, thus overcoming one of the major hurdles to achieving diagnostic excellence. The described program also focuses on harnessing the power of teamwork, which is especially relevant given recent descriptions of the effectiveness of collective intelligence in improving diagnostic performance.⁶ Importantly, the authors recognize that their intervention will not achieve a diagnosis in every case for which they are consulted; rather, they hope that their thorough evaluation will uncover additional potential diagnostic avenues for the referring team to pursue, with a goal to “improve patient care by providing…ideas to reduce—or at least manage—diagnostic uncertainty.”

Programs of this nature are exciting for hospitalists. Hospital medicine is, perhaps, a place in modern medicine where diagnostic excellence has a natural home. Patients admitted to the hospital are acutely (and often severely) ill, and hospitalists are tasked with rapidly identifying the cause of their illness in order to initiate appropriate treatment and accurately inform prognosis. Hospitalists, as generalists, take a broad approach to challenging cases, and they tend to practice in well-resourced environments with nearly every diagnostic modality at their disposal. Many hospitalists would envy participating in a program such as the Socrates Project.

While Singer et al.’s innovation—and the institutional support thereof—should be lauded, some discussion must be had about how to assess the effectiveness of such a program. The authors acknowledge the need for evaluation of both the diagnostic process and the outcomes that process achieves. Measuring diagnostic performance is challenging, however, and while there is substantial progress being made in this area, recent efforts tend to focus on identifying diagnostic errors rather than measuring diagnostic excellence. Moreover, even if a program does improve diagnostic performance, how should we evaluate for unintended consequences of its implementation? In the age of high-value care, how can we ensure that efforts to do a better job of spotting proverbial zebras do not come at the cost of harming too many horses?⁷

Hospitalists are well primed to answer this question. The juxtaposition of Singer et al.’s article with the Journal of Hospital Medicine’s long-running series on Choosing Wisely®: Things We Do for No Reason™ provides a natural synergy to begin crafting a framework to evaluate unintended consequences of a program in diagnostic excellence. More diagnosis is not the goal; more appropriate diagnosis is what is needed. A clinical program aimed at achieving diagnostic excellence should not employ low-value, wasteful strategies that do not add substantively to the diagnostic process but should instead seek to improve the overall efficiency of even complicated diagnostic odysseys. Avoiding waste throughout will allow for allocation of diagnostic resources where they are needed. In turn, hospitalists can do a better job of correctly identifying both horses and zebras for what they are. While a given hospitalization for a diagnostically complex patient may be relatively expensive, better diagnosis during an index hospitalization is likely to lead to decreased downstream costs, such as those related to readmissions and further testing, as well as better health outcomes.

The Socrates Project, along with similar programs at other institutions, are exciting innovations. These programs are not only likely to be good for patients but are also good for hospitalists. The field of hospital medicine should leverage its collective expertise in clinical medicine, systems of care, and high-value care to become a home for diagnostic excellence.

“No one cares how much you know, until they know how much you care.”

—Theodore Roosevelt (attributed)

Like many early career clinician-educators, you are likely embarking on your teaching role with excitement and trepidation. Excitement accompanies the opportunity to develop the next generation of physicians. Trepidation arises from a fear of insufficient knowledge. This concern is understandable but misplaced: great teachers are great because of their emotional intelligence, not their medical intelligence. These five principles will help you establish an optimal learning environment.

Small-Talk before Med-Talk. “What do you like to do outside of the hospital?” “Where is your favorite place to eat?” These questions indicate that your interest in learners transcends clinical work. Leaders who are more relationship- than task-oriented achieve greater group cohesion and more team learning. Exemplary inpatient attending physicians use learners’ first names and get to know them on a personal level to signal that they care as much about the person as they do about the performance.¹

Be Available. Medical educators balance supervision and autonomy while trainees engage in high-stakes decisions. The best teachers get this right by signaling “I have faith in you” and “I’m always available.” Clinician-educator Kimberly Manning, MD portrayed this balance in a recent Twitter thread. The resident called: “I am sorry to bother you.” Dr. Manning responded, “Never be sorry.” The resident was concerned about a patient with new abdominal pain but reassured Dr. Manning that she did not need to return to the hospital. She returned anyway. She assessed the patient and had nothing to add to the resident’s outstanding management. As the patient recovered from his operation for a perforated ulcer, Dr. Manning reflected, “On a perfect Saturday afternoon, I chose to return to the hospital. To make not one decision or write one single order. But instead to stand beside my resident and intentionally affirm her.”

Build from the Ground Up. Asking questions is the teacher’s core procedure. Strive to master the true Socratic method of starting with an elemental inquiry and then leading a conversation that poses questions of increasing difficulty until you reach the limits of the learner’s understanding. This method reinforces their hard-earned knowledge and sets the stage for growth. “What would be your first test to evaluate tachycardia?” Once the correct answer is firmly in hand, explore the margin of their knowledge. “Which regular, narrow complex tachycardias stop with adenosine?”

Never Judge. Never endorse an incorrect response—but do not disparage it either. A trainee must learn that their answer was wrong but should not feel defeated or embarrassed. Use judgment regarding whether constructive feedback should be delivered in public or in private.

I recall answering a question incorrectly in medical school. The attending responded, “How many years did you take off before starting third year?” I had not taken any time off. The attending was a phenomenal clinician but a lousy teacher. A master teacher would have accessed a foothold and built my knowledge without judgment.

Remain Humble. One of the most liberating phrases you will deploy as a teacher is “I don’t know.” Its utterance demonstrates the honesty and humility you hope to instill in learners. Be on the lookout for the many times your trainees will know more than you.

Recently my team evaluated a patient with blunted facial expression, bradykinesia, and a resting hand tremor. I disclosed to my team: “I don’t know the key maneuvers to distinguish the Parkinson plus syndromes from Parkinson disease.” The medical student had spent one year studying patients with neurodegenerative diseases (I learned this during the “small-talk before med-talk” phase). I invited him to demonstrate the neurologic exam, which he did admirably. That day I did not know the subject well, and we all learned because I freely admitted it.

Being a physician is the greatest job in the world. If you leverage your EQ (emotional quotient) as much as your IQ (intelligence quotient), your learners will conclude the same.

“No one cares how much you know, until they know how much you care.”

—Theodore Roosevelt (attributed)

Like many early career clinician-educators, you are likely embarking on your teaching role with excitement and trepidation. Excitement accompanies the opportunity to develop the next generation of physicians. Trepidation arises from a fear of insufficient knowledge. This concern is understandable but misplaced: great teachers are great because of their emotional intelligence, not their medical intelligence. These five principles will help you establish an optimal learning environment.

Small-Talk before Med-Talk. “What do you like to do outside of the hospital?” “Where is your favorite place to eat?” These questions indicate that your interest in learners transcends clinical work. Leaders who are more relationship- than task-oriented achieve greater group cohesion and more team learning. Exemplary inpatient attending physicians use learners’ first names and get to know them on a personal level to signal that they care as much about the person as they do about the performance.¹

Be Available. Medical educators balance supervision and autonomy while trainees engage in high-stakes decisions. The best teachers get this right by signaling “I have faith in you” and “I’m always available.” Clinician-educator Kimberly Manning, MD portrayed this balance in a recent Twitter thread. The resident called: “I am sorry to bother you.” Dr. Manning responded, “Never be sorry.” The resident was concerned about a patient with new abdominal pain but reassured Dr. Manning that she did not need to return to the hospital. She returned anyway. She assessed the patient and had nothing to add to the resident’s outstanding management. As the patient recovered from his operation for a perforated ulcer, Dr. Manning reflected, “On a perfect Saturday afternoon, I chose to return to the hospital. To make not one decision or write one single order. But instead to stand beside my resident and intentionally affirm her.”

Build from the Ground Up. Asking questions is the teacher’s core procedure. Strive to master the true Socratic method of starting with an elemental inquiry and then leading a conversation that poses questions of increasing difficulty until you reach the limits of the learner’s understanding. This method reinforces their hard-earned knowledge and sets the stage for growth. “What would be your first test to evaluate tachycardia?” Once the correct answer is firmly in hand, explore the margin of their knowledge. “Which regular, narrow complex tachycardias stop with adenosine?”

Never Judge. Never endorse an incorrect response—but do not disparage it either. A trainee must learn that their answer was wrong but should not feel defeated or embarrassed. Use judgment regarding whether constructive feedback should be delivered in public or in private.

I recall answering a question incorrectly in medical school. The attending responded, “How many years did you take off before starting third year?” I had not taken any time off. The attending was a phenomenal clinician but a lousy teacher. A master teacher would have accessed a foothold and built my knowledge without judgment.

Remain Humble. One of the most liberating phrases you will deploy as a teacher is “I don’t know.” Its utterance demonstrates the honesty and humility you hope to instill in learners. Be on the lookout for the many times your trainees will know more than you.

Recently my team evaluated a patient with blunted facial expression, bradykinesia, and a resting hand tremor. I disclosed to my team: “I don’t know the key maneuvers to distinguish the Parkinson plus syndromes from Parkinson disease.” The medical student had spent one year studying patients with neurodegenerative diseases (I learned this during the “small-talk before med-talk” phase). I invited him to demonstrate the neurologic exam, which he did admirably. That day I did not know the subject well, and we all learned because I freely admitted it.

Being a physician is the greatest job in the world. If you leverage your EQ (emotional quotient) as much as your IQ (intelligence quotient), your learners will conclude the same.

“No one cares how much you know, until they know how much you care.”

—Theodore Roosevelt (attributed)

Like many early career clinician-educators, you are likely embarking on your teaching role with excitement and trepidation. Excitement accompanies the opportunity to develop the next generation of physicians. Trepidation arises from a fear of insufficient knowledge. This concern is understandable but misplaced: great teachers are great because of their emotional intelligence, not their medical intelligence. These five principles will help you establish an optimal learning environment.

Small-Talk before Med-Talk. “What do you like to do outside of the hospital?” “Where is your favorite place to eat?” These questions indicate that your interest in learners transcends clinical work. Leaders who are more relationship- than task-oriented achieve greater group cohesion and more team learning. Exemplary inpatient attending physicians use learners’ first names and get to know them on a personal level to signal that they care as much about the person as they do about the performance.¹

Be Available. Medical educators balance supervision and autonomy while trainees engage in high-stakes decisions. The best teachers get this right by signaling “I have faith in you” and “I’m always available.” Clinician-educator Kimberly Manning, MD portrayed this balance in a recent Twitter thread. The resident called: “I am sorry to bother you.” Dr. Manning responded, “Never be sorry.” The resident was concerned about a patient with new abdominal pain but reassured Dr. Manning that she did not need to return to the hospital. She returned anyway. She assessed the patient and had nothing to add to the resident’s outstanding management. As the patient recovered from his operation for a perforated ulcer, Dr. Manning reflected, “On a perfect Saturday afternoon, I chose to return to the hospital. To make not one decision or write one single order. But instead to stand beside my resident and intentionally affirm her.”

Build from the Ground Up. Asking questions is the teacher’s core procedure. Strive to master the true Socratic method of starting with an elemental inquiry and then leading a conversation that poses questions of increasing difficulty until you reach the limits of the learner’s understanding. This method reinforces their hard-earned knowledge and sets the stage for growth. “What would be your first test to evaluate tachycardia?” Once the correct answer is firmly in hand, explore the margin of their knowledge. “Which regular, narrow complex tachycardias stop with adenosine?”

Never Judge. Never endorse an incorrect response—but do not disparage it either. A trainee must learn that their answer was wrong but should not feel defeated or embarrassed. Use judgment regarding whether constructive feedback should be delivered in public or in private.

I recall answering a question incorrectly in medical school. The attending responded, “How many years did you take off before starting third year?” I had not taken any time off. The attending was a phenomenal clinician but a lousy teacher. A master teacher would have accessed a foothold and built my knowledge without judgment.

Remain Humble. One of the most liberating phrases you will deploy as a teacher is “I don’t know.” Its utterance demonstrates the honesty and humility you hope to instill in learners. Be on the lookout for the many times your trainees will know more than you.

Recently my team evaluated a patient with blunted facial expression, bradykinesia, and a resting hand tremor. I disclosed to my team: “I don’t know the key maneuvers to distinguish the Parkinson plus syndromes from Parkinson disease.” The medical student had spent one year studying patients with neurodegenerative diseases (I learned this during the “small-talk before med-talk” phase). I invited him to demonstrate the neurologic exam, which he did admirably. That day I did not know the subject well, and we all learned because I freely admitted it.

Being a physician is the greatest job in the world. If you leverage your EQ (emotional quotient) as much as your IQ (intelligence quotient), your learners will conclude the same.

Hyperkalemia (serum potassium ≥5.1 mEq/L), if left untreated, may result in cardiac arrhythmias, severe muscle weakness, or paralysis.^1,2 Insulin administration can rapidly correct hyperkalemia by shifting serum potassiufm intracellularly.³ Treatment of hyperkalemia with insulin may lead to hypoglycemia, which, when severe, can cause confusion, seizures, loss of consciousness, and death. The use of regular and short-acting insulins to correct hyperkalemia quickly in hospitalized patients results in the greatest risk of hypoglycemia within three hours of treatment.⁴ Nonetheless, monitoring blood glucose levels within six hours of postinsulin administration is not a standard part of hyperkalemia treatment guidelines,³ leaving the rates of hypoglycemia in this setting poorly characterized.

Without standardized blood glucose measurement protocols, retrospective studies have reported posttreatment hypoglycemia rates of 8.7%-17.5% among all patients with hyperkalemia,^5,6 and 13% among patients with end-stage renal disease.⁴ These estimates likely underestimate the true hypoglycemia rates as they measure blood glucose sporadically and are often outside the three-hour window of highest risk after insulin administration.

At the University of California, San Francisco Medical Center (UCSFMC), we faced similar issues in measuring the true hypoglycemia rates associated with hyperkalemia treatment. In December 2015, a 12-month retrospective review revealed a 12% hypoglycemia rate among patients treated with insulin for hyperkalemia. This review was limited by the inclusion of only patients treated for hyperkalemia using the standard orderset supplied with the electronic health record system (EHR; EPIC Systems, Verona, Wisconsin) and the absence of specific orders for glucose monitoring. As a result, more than 40% of these inpatients had no documented glucose within six hours of postinsulin administration.

We subsequently designed and implemented an adult inpatient hyperkalemia treatment orderset aimed at reducing iatrogenic hypoglycemia by promoting appropriate insulin use and blood glucose monitoring during the treatment of hyperkalemia. Through rapid improvement cycles, we iteratively revised the orderset to optimally mitigate the risk of hypoglycemia that was associated with insulin use. We describe implementation and outcomes of weight-based insulin dosing,⁷ automated alerts to identify patients at greatest risk for hypoglycemia, and clinical decision support based on the preinsulin blood glucose level. We report the rates of iatrogenic hypoglycemia after the implementation of these order-set changes.

Design Overview

EHR data were extracted from Epic Clarity. We analyzed data following Orderset 1.1 implementation (January 1, 2016-March 19, 2017) when hypoglycemia rates were reliably quantifiable and following orderset revision 1.2 (March 20, 2017-September 30, 2017) to evaluate the impact of the orderset intervention. The data collection was approved by the Institutional Review Board at the University of California, San Francisco.

Additionally, we explored the frequency in which providers ordered insulin through the hyperkalemia orderset for each version of the orderset via two-month baseline reviews. Investigation for Orderset 1.1 was from January 1, 2017 to February 28, 2017 and for Orderset 1.2 was from August 1, 2017 to September 30, 2017. Insulin ordering frequency through the hyperkalemia orderset was defined as ordering insulin through the adult inpatient hyperkalemia orderset versus ordering insulin in and outside of the hyperkalemia orderset.

Last, we measured the nursing point of care testing (POCT) blood glucose measurement compliance with the hyperkalemia orderset. Nursing utilization acceptance of the hyperkalemia orderset was defined as adequate POCT blood glucose levels monitored in comparison to all insulin treatments via the hyperkalemia orderset.

Setting and Participants

We evaluated nonobstetric adult inpatients admitted to UCSF Medical Center between January 2016 and September 2017. All medical and surgical wards and intensive care units were included in the analysis.

Intervention

In June 2012, an EHR developed by Epic Systems was introduced at UCSFMC. In January 2016, we designed a new EHR-based hyperkalemia treatment orderset (Orderset 1.1), which added standard POCT blood glucose checks before and at one, two, four, and six hours after insulin injection (Appendix 1). In March 2017, a newly designed orderset (Orderset 1.2) replaced the previous hyperkalemia treatment orderset (Appendix 2). Orderset 1.2 included three updates. First, providers were now presented the option of ordering insulin as a weight-based dose (0.1 units/kg intravenous bolus of regular insulin) instead of the previously standard 10 units. Next, provider alerts identifying high-risk patients were built into the EHR. Last, the orderset included tools to guide decision-making based on the preinsulin blood glucose as follows: (1) If preinsulin blood glucose is less than 150 mg/dL, then add an additional dextrose 50% (50 mL) IV once one hour postinsulin administration, and (2) if preinsulin blood glucose is greater than 300 mg/dL, then remove dextrose 50% (50 mL) with insulin administration.

CORRECTED FIGURE PER ERRATUM

Inclusion and exclusion criteria are shown in the Figure. All patients who had insulin ordered via a hyperkalemia orderset were included in an intention-to-treat analysis. A further analysis was performed for patients for whom orderset compliance was achieved (ie, insulin ordered through the ordersets with adequate blood glucose monitoring). These patients were required to have a POCT blood glucose check preinsulin administration and postinsulin administration as follows: (1) between 30 to 180 minutes (0.5 to three hours) after insulin administration, and (2) between 180 and 360 minutes (three to six hours) after insulin administration. For patients receiving repeated insulin treatments for hyperkalemia within six hours, the first treatment data points were excluded to prevent duplication.

Outcomes

We extracted data on all nonobstetric adult patients admitted to UCSFMC between January 1, 2016 and March 19, 2017 (Orderset 1.1) and between March 20, 2017 and September 30, 2017 (Orderset 1.2).

We measured unique insulin administrations given that each insulin injection poses a risk of iatrogenic hypoglycemia. Hypoglycemia was defined as glucose <70 mg/dL and severe hypoglycemia was defined as glucose <40 mg/dL. Covariates included time and date of insulin administration; blood glucose levels before and at one, two, four, and six hours after insulin injection (if available); sex; weight; dose of insulin given for hyperkalemia treatment; creatinine; known diagnosis of diabetes; concomitant use of albuterol; and concomitant use of corticosteroids. Hyperglycemia was defined as glucose >180 mg/dL. We collected potassium levels pre- and postinsulin treatment. The responsible team’s discipline and the location of the patient (eg, medical/surgical unit, intensive care unit, emergency department) where the insulin orderset was used were recorded.

Statistical Analysis

Statistical analysis for our data included the χ2 test for categorical data and Student t test for continuous data. The bivariate analysis identified potential risk factors and protective factors for hypoglycemia, and logistic regression was used to determine independent predictors of hypoglycemia. Through bivariate analyses, any factor with a P value below .05 was included in the multivariable analyses to investigate a significant contribution to hypoglycemia outcomes. Analyses for hypoglycemia and severe hypoglycemia rates, potassium levels pre- and postinsulin treatment, and hyperglycemia rates were done for both the intention-to-treat group and the group with all criteria met. All analyses were rendered utilizing Stata version 14 (Stata Corp LLC, College Station, Texas).

Baseline patient characteristics, initial insulin dosing, the treatment team, and the location are shown in Table 1. With the implementation of weight-based dosing, a lower dose of insulin was administered with Orderset 1.2 compared with Orderset 1.1.

Orderset adherence rates for Orderset 1.1 and 1.2 were as follows: Acute Care Floor 65% (70%), Intensive Care Unit 63% (66%), and Emergency Department 60% (55%). A two-month audit of orderset usage and compliance revealed that 73% (70 of 96) of insulin treatments were ordered through Orderset 1.1, and 77% (71 of 92) of insulin treatments were ordered through Orderset 1.2. The distribution of orderset usage across location and primary service are shown in Table 1.

The patient distribution is shown in the Figure. In the Orderset 1.1 period, there were 352 total insulin treatments utilizing the newly revised UCSFMC adult inpatient hyperkalemia orderset that were used for the intention-to-treat analysis, and there were 225 patients for whom compliance with orderset monitoring was achieved. Notably, 112 treatments were excluded for the lack of adequate blood glucose monitoring. In the Orderset 1.2 period, there were 239 total insulin treatments utilizing the newly revised UCSFMC adult inpatient hyperkalemia orderset that were used for the intention-to-treat analysis, and there were 145 patients for whom compliance with orderset monitoring was achieved. During this phase, 80 treatments were excluded for inadequate blood glucose monitoring.

Treatment of hyperkalemia with insulin may result in significant iatrogenic hypoglycemia. Prior studies have likely underestimated the incidence of hyperkalemia treatment-associated hypoglycemia as glucose levels are rarely checked within three hours of insulin administration.⁸ In our study, which was designed to ensure appropriate blood glucose measurement, 21% of insulin treatments for hyperkalemia resulted in hypoglycemia, with 92% of hypoglycemic events occurring within the first three hours.

For the Orderset 1.1 period, patient risk factors identified for iatrogenic hypoglycemia postinsulin administration were female sex, doses of regular insulin greater than 0.14 units/kg, preinsulin blood glucose less than 140 mg/dL, and serum creatinine greater than 2.5 mg/dL. These results are consistent with studies suggesting that preinsulin blood glucose levels less than 140 mg/dL and the standard 10 units of insulin for hyperkalemia treatment may increase the risk of hypoglycemia.^4,7,9

To decrease the risk of iatrogenic hypoglycemia, we redesigned our hyperkalemia insulin orderset to address the strongest predictors of hypoglycemia (doses of regular insulin greater than 0.14 units/kg and preinsulin blood glucose less than 140 mg/dL). The main changes were weight-based insulin dosing (based on previously published data)¹⁰ and adjustment of glucose administration based on the patient’s glucose levels.¹¹ Following these changes, the rates of both hypoglycemia and severe hypoglycemia were statistically significantly reduced. In addition, of the 14 hypoglycemia events identified after the introduction of Orderset 1.2, five could have been prevented (36%) had the protocol been strictly followed. These five hypoglycemia events occurred later than one-hour postinsulin administration in patients with blood sugars < 150 mg/dL prior to insulin administration. In each of these cases, Orderset 1.2 called for an additional dextrose 50% (50 mL) IV bolus, which likely would have prevented the subsequently recorded hypoglycemia. In other words, our orderset indicated that these patients received an additional bolus of dextrose. However, they did not receive their glucose at the appropriate time, contributing to the hypoglycemia events. The orderset did not include a best practice alert (BPA) to remind providers about the extra dextrose bolus. In the future, we plan to add this BPA.

The hypoglycemia rate identified by Orderset 1.1 was 21% and the hypoglycemia rate identified by the Orderset 1.2 was 10%. The severe hypoglycemia rate identified by Orderset 1.1 was 5% and the severe hypoglycemia rate identified by Orderset 1.2 was 2%. The hypoglycemia and severe hypoglycemia rates significantly decreased after the introduction of Orderset 1.2. To mimic a real-world clinical setting, where monitoring of blood glucose is not always achieved multiple times within a six-hour timeframe of postinsulin treatment for hyperkalemia, we conducted an intention-to-treat analysis. Even when including patients for whom full blood glucose monitoring was not achieved, the introduction of Orderset 1.2 was associated with a significant decrease in the hypoglycemia rate.

To demonstrate whether weight-based dosing of insulin was as effective as a standard dose for hyperkalemia treatment, we compared the impact of Orderset 1.1, which only had the option for single standard doses of insulin, with the impact of Orderset 1.2, which included weight-based dosing options. With the introduction of Orderset 1.2, there was a significant decrease in serum potassium, indicating that weight-based dosing options may not only prevent hypoglycemia but may potentially provide more effective hyperkalemia treatment.

We also compared the rate of hyperglycemia (a glucose >180 mg/dl) pre- and posttreatment (Table 3). Although not statistically significant, the rate of hyperglycemia decreased from 11% to 6%, suggesting a trend toward decreased hyperglycemia with orderset usage.

As orderset usage for hyperkalemia management only occurred approximately 75% of the time, likely, forcing the use of these ordersets would further reduce the incidence of treatment-associated hypoglycemia. To encourage the use of ordersets for hyperkalemia management, our medical center has largely restricted insulin ordering so that it can only be done through ordersets with the proper precautions in place, regardless of the indication. Furthermore, adherence to all the blood glucose monitoring orders embedded in the ordersets remained suboptimal irrespective of managing the service or clinical setting. While we believe that 100% of postglucose monitoring should be possible with appropriate education and institutional support, in some clinical environments, checking glucose levels at least twice in a six-hour window (postinsulin treatment) might be prohibitive. Since 92% of hypoglycemic events occurred within the first three hours postinsulin administration, checking glucose prior to insulin administration and within the first four hours following insulin administration should be prioritized.

Finally, development and implementation of these hyperkalemia treatment ordersets required an experienced multidisciplinary team, including pharmacists, nurses, hospitalists, endocrinologists, and EHR system programmers.^12,13 We, therefore, encourage interprofessional collaboration for any institutions seeking to establish innovative clinical protocols.

This analysis was limited to the insulin administration meeting our inclusion criteria. Thus, we could not identify a true hypoglycemia rate for treatments that were not followed by adequate blood glucose monitoring postinsulin administration, or for insulin administration ordered outside of the hyperkalemia ordersets.

The use of a comprehensive EHR orderset for the treatment of hyperkalemia with predefined times for blood glucose monitoring, weight-based insulin dosing, and prompts to warn providers of an individual patient’s risk for hypoglycemia may significantly reduce the incidence of iatrogenic hypoglycemia.

Hyperkalemia (serum potassium ≥5.1 mEq/L), if left untreated, may result in cardiac arrhythmias, severe muscle weakness, or paralysis.^1,2 Insulin administration can rapidly correct hyperkalemia by shifting serum potassiufm intracellularly.³ Treatment of hyperkalemia with insulin may lead to hypoglycemia, which, when severe, can cause confusion, seizures, loss of consciousness, and death. The use of regular and short-acting insulins to correct hyperkalemia quickly in hospitalized patients results in the greatest risk of hypoglycemia within three hours of treatment.⁴ Nonetheless, monitoring blood glucose levels within six hours of postinsulin administration is not a standard part of hyperkalemia treatment guidelines,³ leaving the rates of hypoglycemia in this setting poorly characterized.

Without standardized blood glucose measurement protocols, retrospective studies have reported posttreatment hypoglycemia rates of 8.7%-17.5% among all patients with hyperkalemia,^5,6 and 13% among patients with end-stage renal disease.⁴ These estimates likely underestimate the true hypoglycemia rates as they measure blood glucose sporadically and are often outside the three-hour window of highest risk after insulin administration.

At the University of California, San Francisco Medical Center (UCSFMC), we faced similar issues in measuring the true hypoglycemia rates associated with hyperkalemia treatment. In December 2015, a 12-month retrospective review revealed a 12% hypoglycemia rate among patients treated with insulin for hyperkalemia. This review was limited by the inclusion of only patients treated for hyperkalemia using the standard orderset supplied with the electronic health record system (EHR; EPIC Systems, Verona, Wisconsin) and the absence of specific orders for glucose monitoring. As a result, more than 40% of these inpatients had no documented glucose within six hours of postinsulin administration.

We subsequently designed and implemented an adult inpatient hyperkalemia treatment orderset aimed at reducing iatrogenic hypoglycemia by promoting appropriate insulin use and blood glucose monitoring during the treatment of hyperkalemia. Through rapid improvement cycles, we iteratively revised the orderset to optimally mitigate the risk of hypoglycemia that was associated with insulin use. We describe implementation and outcomes of weight-based insulin dosing,⁷ automated alerts to identify patients at greatest risk for hypoglycemia, and clinical decision support based on the preinsulin blood glucose level. We report the rates of iatrogenic hypoglycemia after the implementation of these order-set changes.

Design Overview

EHR data were extracted from Epic Clarity. We analyzed data following Orderset 1.1 implementation (January 1, 2016-March 19, 2017) when hypoglycemia rates were reliably quantifiable and following orderset revision 1.2 (March 20, 2017-September 30, 2017) to evaluate the impact of the orderset intervention. The data collection was approved by the Institutional Review Board at the University of California, San Francisco.

Additionally, we explored the frequency in which providers ordered insulin through the hyperkalemia orderset for each version of the orderset via two-month baseline reviews. Investigation for Orderset 1.1 was from January 1, 2017 to February 28, 2017 and for Orderset 1.2 was from August 1, 2017 to September 30, 2017. Insulin ordering frequency through the hyperkalemia orderset was defined as ordering insulin through the adult inpatient hyperkalemia orderset versus ordering insulin in and outside of the hyperkalemia orderset.

Last, we measured the nursing point of care testing (POCT) blood glucose measurement compliance with the hyperkalemia orderset. Nursing utilization acceptance of the hyperkalemia orderset was defined as adequate POCT blood glucose levels monitored in comparison to all insulin treatments via the hyperkalemia orderset.

Setting and Participants

We evaluated nonobstetric adult inpatients admitted to UCSF Medical Center between January 2016 and September 2017. All medical and surgical wards and intensive care units were included in the analysis.

Intervention

In June 2012, an EHR developed by Epic Systems was introduced at UCSFMC. In January 2016, we designed a new EHR-based hyperkalemia treatment orderset (Orderset 1.1), which added standard POCT blood glucose checks before and at one, two, four, and six hours after insulin injection (Appendix 1). In March 2017, a newly designed orderset (Orderset 1.2) replaced the previous hyperkalemia treatment orderset (Appendix 2). Orderset 1.2 included three updates. First, providers were now presented the option of ordering insulin as a weight-based dose (0.1 units/kg intravenous bolus of regular insulin) instead of the previously standard 10 units. Next, provider alerts identifying high-risk patients were built into the EHR. Last, the orderset included tools to guide decision-making based on the preinsulin blood glucose as follows: (1) If preinsulin blood glucose is less than 150 mg/dL, then add an additional dextrose 50% (50 mL) IV once one hour postinsulin administration, and (2) if preinsulin blood glucose is greater than 300 mg/dL, then remove dextrose 50% (50 mL) with insulin administration.

CORRECTED FIGURE PER ERRATUM

Inclusion and exclusion criteria are shown in the Figure. All patients who had insulin ordered via a hyperkalemia orderset were included in an intention-to-treat analysis. A further analysis was performed for patients for whom orderset compliance was achieved (ie, insulin ordered through the ordersets with adequate blood glucose monitoring). These patients were required to have a POCT blood glucose check preinsulin administration and postinsulin administration as follows: (1) between 30 to 180 minutes (0.5 to three hours) after insulin administration, and (2) between 180 and 360 minutes (three to six hours) after insulin administration. For patients receiving repeated insulin treatments for hyperkalemia within six hours, the first treatment data points were excluded to prevent duplication.

Outcomes

We extracted data on all nonobstetric adult patients admitted to UCSFMC between January 1, 2016 and March 19, 2017 (Orderset 1.1) and between March 20, 2017 and September 30, 2017 (Orderset 1.2).

We measured unique insulin administrations given that each insulin injection poses a risk of iatrogenic hypoglycemia. Hypoglycemia was defined as glucose <70 mg/dL and severe hypoglycemia was defined as glucose <40 mg/dL. Covariates included time and date of insulin administration; blood glucose levels before and at one, two, four, and six hours after insulin injection (if available); sex; weight; dose of insulin given for hyperkalemia treatment; creatinine; known diagnosis of diabetes; concomitant use of albuterol; and concomitant use of corticosteroids. Hyperglycemia was defined as glucose >180 mg/dL. We collected potassium levels pre- and postinsulin treatment. The responsible team’s discipline and the location of the patient (eg, medical/surgical unit, intensive care unit, emergency department) where the insulin orderset was used were recorded.

Statistical Analysis

Statistical analysis for our data included the χ2 test for categorical data and Student t test for continuous data. The bivariate analysis identified potential risk factors and protective factors for hypoglycemia, and logistic regression was used to determine independent predictors of hypoglycemia. Through bivariate analyses, any factor with a P value below .05 was included in the multivariable analyses to investigate a significant contribution to hypoglycemia outcomes. Analyses for hypoglycemia and severe hypoglycemia rates, potassium levels pre- and postinsulin treatment, and hyperglycemia rates were done for both the intention-to-treat group and the group with all criteria met. All analyses were rendered utilizing Stata version 14 (Stata Corp LLC, College Station, Texas).

Baseline patient characteristics, initial insulin dosing, the treatment team, and the location are shown in Table 1. With the implementation of weight-based dosing, a lower dose of insulin was administered with Orderset 1.2 compared with Orderset 1.1.

Orderset adherence rates for Orderset 1.1 and 1.2 were as follows: Acute Care Floor 65% (70%), Intensive Care Unit 63% (66%), and Emergency Department 60% (55%). A two-month audit of orderset usage and compliance revealed that 73% (70 of 96) of insulin treatments were ordered through Orderset 1.1, and 77% (71 of 92) of insulin treatments were ordered through Orderset 1.2. The distribution of orderset usage across location and primary service are shown in Table 1.

The patient distribution is shown in the Figure. In the Orderset 1.1 period, there were 352 total insulin treatments utilizing the newly revised UCSFMC adult inpatient hyperkalemia orderset that were used for the intention-to-treat analysis, and there were 225 patients for whom compliance with orderset monitoring was achieved. Notably, 112 treatments were excluded for the lack of adequate blood glucose monitoring. In the Orderset 1.2 period, there were 239 total insulin treatments utilizing the newly revised UCSFMC adult inpatient hyperkalemia orderset that were used for the intention-to-treat analysis, and there were 145 patients for whom compliance with orderset monitoring was achieved. During this phase, 80 treatments were excluded for inadequate blood glucose monitoring.

Treatment of hyperkalemia with insulin may result in significant iatrogenic hypoglycemia. Prior studies have likely underestimated the incidence of hyperkalemia treatment-associated hypoglycemia as glucose levels are rarely checked within three hours of insulin administration.⁸ In our study, which was designed to ensure appropriate blood glucose measurement, 21% of insulin treatments for hyperkalemia resulted in hypoglycemia, with 92% of hypoglycemic events occurring within the first three hours.

For the Orderset 1.1 period, patient risk factors identified for iatrogenic hypoglycemia postinsulin administration were female sex, doses of regular insulin greater than 0.14 units/kg, preinsulin blood glucose less than 140 mg/dL, and serum creatinine greater than 2.5 mg/dL. These results are consistent with studies suggesting that preinsulin blood glucose levels less than 140 mg/dL and the standard 10 units of insulin for hyperkalemia treatment may increase the risk of hypoglycemia.^4,7,9

To decrease the risk of iatrogenic hypoglycemia, we redesigned our hyperkalemia insulin orderset to address the strongest predictors of hypoglycemia (doses of regular insulin greater than 0.14 units/kg and preinsulin blood glucose less than 140 mg/dL). The main changes were weight-based insulin dosing (based on previously published data)¹⁰ and adjustment of glucose administration based on the patient’s glucose levels.¹¹ Following these changes, the rates of both hypoglycemia and severe hypoglycemia were statistically significantly reduced. In addition, of the 14 hypoglycemia events identified after the introduction of Orderset 1.2, five could have been prevented (36%) had the protocol been strictly followed. These five hypoglycemia events occurred later than one-hour postinsulin administration in patients with blood sugars < 150 mg/dL prior to insulin administration. In each of these cases, Orderset 1.2 called for an additional dextrose 50% (50 mL) IV bolus, which likely would have prevented the subsequently recorded hypoglycemia. In other words, our orderset indicated that these patients received an additional bolus of dextrose. However, they did not receive their glucose at the appropriate time, contributing to the hypoglycemia events. The orderset did not include a best practice alert (BPA) to remind providers about the extra dextrose bolus. In the future, we plan to add this BPA.

The hypoglycemia rate identified by Orderset 1.1 was 21% and the hypoglycemia rate identified by the Orderset 1.2 was 10%. The severe hypoglycemia rate identified by Orderset 1.1 was 5% and the severe hypoglycemia rate identified by Orderset 1.2 was 2%. The hypoglycemia and severe hypoglycemia rates significantly decreased after the introduction of Orderset 1.2. To mimic a real-world clinical setting, where monitoring of blood glucose is not always achieved multiple times within a six-hour timeframe of postinsulin treatment for hyperkalemia, we conducted an intention-to-treat analysis. Even when including patients for whom full blood glucose monitoring was not achieved, the introduction of Orderset 1.2 was associated with a significant decrease in the hypoglycemia rate.

To demonstrate whether weight-based dosing of insulin was as effective as a standard dose for hyperkalemia treatment, we compared the impact of Orderset 1.1, which only had the option for single standard doses of insulin, with the impact of Orderset 1.2, which included weight-based dosing options. With the introduction of Orderset 1.2, there was a significant decrease in serum potassium, indicating that weight-based dosing options may not only prevent hypoglycemia but may potentially provide more effective hyperkalemia treatment.

We also compared the rate of hyperglycemia (a glucose >180 mg/dl) pre- and posttreatment (Table 3). Although not statistically significant, the rate of hyperglycemia decreased from 11% to 6%, suggesting a trend toward decreased hyperglycemia with orderset usage.

As orderset usage for hyperkalemia management only occurred approximately 75% of the time, likely, forcing the use of these ordersets would further reduce the incidence of treatment-associated hypoglycemia. To encourage the use of ordersets for hyperkalemia management, our medical center has largely restricted insulin ordering so that it can only be done through ordersets with the proper precautions in place, regardless of the indication. Furthermore, adherence to all the blood glucose monitoring orders embedded in the ordersets remained suboptimal irrespective of managing the service or clinical setting. While we believe that 100% of postglucose monitoring should be possible with appropriate education and institutional support, in some clinical environments, checking glucose levels at least twice in a six-hour window (postinsulin treatment) might be prohibitive. Since 92% of hypoglycemic events occurred within the first three hours postinsulin administration, checking glucose prior to insulin administration and within the first four hours following insulin administration should be prioritized.

Finally, development and implementation of these hyperkalemia treatment ordersets required an experienced multidisciplinary team, including pharmacists, nurses, hospitalists, endocrinologists, and EHR system programmers.^12,13 We, therefore, encourage interprofessional collaboration for any institutions seeking to establish innovative clinical protocols.

This analysis was limited to the insulin administration meeting our inclusion criteria. Thus, we could not identify a true hypoglycemia rate for treatments that were not followed by adequate blood glucose monitoring postinsulin administration, or for insulin administration ordered outside of the hyperkalemia ordersets.

The use of a comprehensive EHR orderset for the treatment of hyperkalemia with predefined times for blood glucose monitoring, weight-based insulin dosing, and prompts to warn providers of an individual patient’s risk for hypoglycemia may significantly reduce the incidence of iatrogenic hypoglycemia.

Hyperkalemia (serum potassium ≥5.1 mEq/L), if left untreated, may result in cardiac arrhythmias, severe muscle weakness, or paralysis.^1,2 Insulin administration can rapidly correct hyperkalemia by shifting serum potassiufm intracellularly.³ Treatment of hyperkalemia with insulin may lead to hypoglycemia, which, when severe, can cause confusion, seizures, loss of consciousness, and death. The use of regular and short-acting insulins to correct hyperkalemia quickly in hospitalized patients results in the greatest risk of hypoglycemia within three hours of treatment.⁴ Nonetheless, monitoring blood glucose levels within six hours of postinsulin administration is not a standard part of hyperkalemia treatment guidelines,³ leaving the rates of hypoglycemia in this setting poorly characterized.

Without standardized blood glucose measurement protocols, retrospective studies have reported posttreatment hypoglycemia rates of 8.7%-17.5% among all patients with hyperkalemia,^5,6 and 13% among patients with end-stage renal disease.⁴ These estimates likely underestimate the true hypoglycemia rates as they measure blood glucose sporadically and are often outside the three-hour window of highest risk after insulin administration.

At the University of California, San Francisco Medical Center (UCSFMC), we faced similar issues in measuring the true hypoglycemia rates associated with hyperkalemia treatment. In December 2015, a 12-month retrospective review revealed a 12% hypoglycemia rate among patients treated with insulin for hyperkalemia. This review was limited by the inclusion of only patients treated for hyperkalemia using the standard orderset supplied with the electronic health record system (EHR; EPIC Systems, Verona, Wisconsin) and the absence of specific orders for glucose monitoring. As a result, more than 40% of these inpatients had no documented glucose within six hours of postinsulin administration.

We subsequently designed and implemented an adult inpatient hyperkalemia treatment orderset aimed at reducing iatrogenic hypoglycemia by promoting appropriate insulin use and blood glucose monitoring during the treatment of hyperkalemia. Through rapid improvement cycles, we iteratively revised the orderset to optimally mitigate the risk of hypoglycemia that was associated with insulin use. We describe implementation and outcomes of weight-based insulin dosing,⁷ automated alerts to identify patients at greatest risk for hypoglycemia, and clinical decision support based on the preinsulin blood glucose level. We report the rates of iatrogenic hypoglycemia after the implementation of these order-set changes.

Design Overview

EHR data were extracted from Epic Clarity. We analyzed data following Orderset 1.1 implementation (January 1, 2016-March 19, 2017) when hypoglycemia rates were reliably quantifiable and following orderset revision 1.2 (March 20, 2017-September 30, 2017) to evaluate the impact of the orderset intervention. The data collection was approved by the Institutional Review Board at the University of California, San Francisco.

Additionally, we explored the frequency in which providers ordered insulin through the hyperkalemia orderset for each version of the orderset via two-month baseline reviews. Investigation for Orderset 1.1 was from January 1, 2017 to February 28, 2017 and for Orderset 1.2 was from August 1, 2017 to September 30, 2017. Insulin ordering frequency through the hyperkalemia orderset was defined as ordering insulin through the adult inpatient hyperkalemia orderset versus ordering insulin in and outside of the hyperkalemia orderset.

Last, we measured the nursing point of care testing (POCT) blood glucose measurement compliance with the hyperkalemia orderset. Nursing utilization acceptance of the hyperkalemia orderset was defined as adequate POCT blood glucose levels monitored in comparison to all insulin treatments via the hyperkalemia orderset.

Setting and Participants

We evaluated nonobstetric adult inpatients admitted to UCSF Medical Center between January 2016 and September 2017. All medical and surgical wards and intensive care units were included in the analysis.

Intervention

In June 2012, an EHR developed by Epic Systems was introduced at UCSFMC. In January 2016, we designed a new EHR-based hyperkalemia treatment orderset (Orderset 1.1), which added standard POCT blood glucose checks before and at one, two, four, and six hours after insulin injection (Appendix 1). In March 2017, a newly designed orderset (Orderset 1.2) replaced the previous hyperkalemia treatment orderset (Appendix 2). Orderset 1.2 included three updates. First, providers were now presented the option of ordering insulin as a weight-based dose (0.1 units/kg intravenous bolus of regular insulin) instead of the previously standard 10 units. Next, provider alerts identifying high-risk patients were built into the EHR. Last, the orderset included tools to guide decision-making based on the preinsulin blood glucose as follows: (1) If preinsulin blood glucose is less than 150 mg/dL, then add an additional dextrose 50% (50 mL) IV once one hour postinsulin administration, and (2) if preinsulin blood glucose is greater than 300 mg/dL, then remove dextrose 50% (50 mL) with insulin administration.

CORRECTED FIGURE PER ERRATUM

Inclusion and exclusion criteria are shown in the Figure. All patients who had insulin ordered via a hyperkalemia orderset were included in an intention-to-treat analysis. A further analysis was performed for patients for whom orderset compliance was achieved (ie, insulin ordered through the ordersets with adequate blood glucose monitoring). These patients were required to have a POCT blood glucose check preinsulin administration and postinsulin administration as follows: (1) between 30 to 180 minutes (0.5 to three hours) after insulin administration, and (2) between 180 and 360 minutes (three to six hours) after insulin administration. For patients receiving repeated insulin treatments for hyperkalemia within six hours, the first treatment data points were excluded to prevent duplication.

Outcomes

We extracted data on all nonobstetric adult patients admitted to UCSFMC between January 1, 2016 and March 19, 2017 (Orderset 1.1) and between March 20, 2017 and September 30, 2017 (Orderset 1.2).

We measured unique insulin administrations given that each insulin injection poses a risk of iatrogenic hypoglycemia. Hypoglycemia was defined as glucose <70 mg/dL and severe hypoglycemia was defined as glucose <40 mg/dL. Covariates included time and date of insulin administration; blood glucose levels before and at one, two, four, and six hours after insulin injection (if available); sex; weight; dose of insulin given for hyperkalemia treatment; creatinine; known diagnosis of diabetes; concomitant use of albuterol; and concomitant use of corticosteroids. Hyperglycemia was defined as glucose >180 mg/dL. We collected potassium levels pre- and postinsulin treatment. The responsible team’s discipline and the location of the patient (eg, medical/surgical unit, intensive care unit, emergency department) where the insulin orderset was used were recorded.

Statistical Analysis

Statistical analysis for our data included the χ2 test for categorical data and Student t test for continuous data. The bivariate analysis identified potential risk factors and protective factors for hypoglycemia, and logistic regression was used to determine independent predictors of hypoglycemia. Through bivariate analyses, any factor with a P value below .05 was included in the multivariable analyses to investigate a significant contribution to hypoglycemia outcomes. Analyses for hypoglycemia and severe hypoglycemia rates, potassium levels pre- and postinsulin treatment, and hyperglycemia rates were done for both the intention-to-treat group and the group with all criteria met. All analyses were rendered utilizing Stata version 14 (Stata Corp LLC, College Station, Texas).

Baseline patient characteristics, initial insulin dosing, the treatment team, and the location are shown in Table 1. With the implementation of weight-based dosing, a lower dose of insulin was administered with Orderset 1.2 compared with Orderset 1.1.

Orderset adherence rates for Orderset 1.1 and 1.2 were as follows: Acute Care Floor 65% (70%), Intensive Care Unit 63% (66%), and Emergency Department 60% (55%). A two-month audit of orderset usage and compliance revealed that 73% (70 of 96) of insulin treatments were ordered through Orderset 1.1, and 77% (71 of 92) of insulin treatments were ordered through Orderset 1.2. The distribution of orderset usage across location and primary service are shown in Table 1.

The patient distribution is shown in the Figure. In the Orderset 1.1 period, there were 352 total insulin treatments utilizing the newly revised UCSFMC adult inpatient hyperkalemia orderset that were used for the intention-to-treat analysis, and there were 225 patients for whom compliance with orderset monitoring was achieved. Notably, 112 treatments were excluded for the lack of adequate blood glucose monitoring. In the Orderset 1.2 period, there were 239 total insulin treatments utilizing the newly revised UCSFMC adult inpatient hyperkalemia orderset that were used for the intention-to-treat analysis, and there were 145 patients for whom compliance with orderset monitoring was achieved. During this phase, 80 treatments were excluded for inadequate blood glucose monitoring.

Treatment of hyperkalemia with insulin may result in significant iatrogenic hypoglycemia. Prior studies have likely underestimated the incidence of hyperkalemia treatment-associated hypoglycemia as glucose levels are rarely checked within three hours of insulin administration.⁸ In our study, which was designed to ensure appropriate blood glucose measurement, 21% of insulin treatments for hyperkalemia resulted in hypoglycemia, with 92% of hypoglycemic events occurring within the first three hours.

For the Orderset 1.1 period, patient risk factors identified for iatrogenic hypoglycemia postinsulin administration were female sex, doses of regular insulin greater than 0.14 units/kg, preinsulin blood glucose less than 140 mg/dL, and serum creatinine greater than 2.5 mg/dL. These results are consistent with studies suggesting that preinsulin blood glucose levels less than 140 mg/dL and the standard 10 units of insulin for hyperkalemia treatment may increase the risk of hypoglycemia.^4,7,9

To decrease the risk of iatrogenic hypoglycemia, we redesigned our hyperkalemia insulin orderset to address the strongest predictors of hypoglycemia (doses of regular insulin greater than 0.14 units/kg and preinsulin blood glucose less than 140 mg/dL). The main changes were weight-based insulin dosing (based on previously published data)¹⁰ and adjustment of glucose administration based on the patient’s glucose levels.¹¹ Following these changes, the rates of both hypoglycemia and severe hypoglycemia were statistically significantly reduced. In addition, of the 14 hypoglycemia events identified after the introduction of Orderset 1.2, five could have been prevented (36%) had the protocol been strictly followed. These five hypoglycemia events occurred later than one-hour postinsulin administration in patients with blood sugars < 150 mg/dL prior to insulin administration. In each of these cases, Orderset 1.2 called for an additional dextrose 50% (50 mL) IV bolus, which likely would have prevented the subsequently recorded hypoglycemia. In other words, our orderset indicated that these patients received an additional bolus of dextrose. However, they did not receive their glucose at the appropriate time, contributing to the hypoglycemia events. The orderset did not include a best practice alert (BPA) to remind providers about the extra dextrose bolus. In the future, we plan to add this BPA.

The hypoglycemia rate identified by Orderset 1.1 was 21% and the hypoglycemia rate identified by the Orderset 1.2 was 10%. The severe hypoglycemia rate identified by Orderset 1.1 was 5% and the severe hypoglycemia rate identified by Orderset 1.2 was 2%. The hypoglycemia and severe hypoglycemia rates significantly decreased after the introduction of Orderset 1.2. To mimic a real-world clinical setting, where monitoring of blood glucose is not always achieved multiple times within a six-hour timeframe of postinsulin treatment for hyperkalemia, we conducted an intention-to-treat analysis. Even when including patients for whom full blood glucose monitoring was not achieved, the introduction of Orderset 1.2 was associated with a significant decrease in the hypoglycemia rate.

To demonstrate whether weight-based dosing of insulin was as effective as a standard dose for hyperkalemia treatment, we compared the impact of Orderset 1.1, which only had the option for single standard doses of insulin, with the impact of Orderset 1.2, which included weight-based dosing options. With the introduction of Orderset 1.2, there was a significant decrease in serum potassium, indicating that weight-based dosing options may not only prevent hypoglycemia but may potentially provide more effective hyperkalemia treatment.

We also compared the rate of hyperglycemia (a glucose >180 mg/dl) pre- and posttreatment (Table 3). Although not statistically significant, the rate of hyperglycemia decreased from 11% to 6%, suggesting a trend toward decreased hyperglycemia with orderset usage.

As orderset usage for hyperkalemia management only occurred approximately 75% of the time, likely, forcing the use of these ordersets would further reduce the incidence of treatment-associated hypoglycemia. To encourage the use of ordersets for hyperkalemia management, our medical center has largely restricted insulin ordering so that it can only be done through ordersets with the proper precautions in place, regardless of the indication. Furthermore, adherence to all the blood glucose monitoring orders embedded in the ordersets remained suboptimal irrespective of managing the service or clinical setting. While we believe that 100% of postglucose monitoring should be possible with appropriate education and institutional support, in some clinical environments, checking glucose levels at least twice in a six-hour window (postinsulin treatment) might be prohibitive. Since 92% of hypoglycemic events occurred within the first three hours postinsulin administration, checking glucose prior to insulin administration and within the first four hours following insulin administration should be prioritized.

Finally, development and implementation of these hyperkalemia treatment ordersets required an experienced multidisciplinary team, including pharmacists, nurses, hospitalists, endocrinologists, and EHR system programmers.^12,13 We, therefore, encourage interprofessional collaboration for any institutions seeking to establish innovative clinical protocols.

This analysis was limited to the insulin administration meeting our inclusion criteria. Thus, we could not identify a true hypoglycemia rate for treatments that were not followed by adequate blood glucose monitoring postinsulin administration, or for insulin administration ordered outside of the hyperkalemia ordersets.

The use of a comprehensive EHR orderset for the treatment of hyperkalemia with predefined times for blood glucose monitoring, weight-based insulin dosing, and prompts to warn providers of an individual patient’s risk for hypoglycemia may significantly reduce the incidence of iatrogenic hypoglycemia.

The Veterans Health Administration (VHA) provides care for 9 million veterans at 1,255 health care sites linked to one of 170 local medical systems.¹ Recognizing that providing timely care requires effective access management, the US Congress mandated training of VHA staff to manage and improve access to care but did not provide additional local funds for new positions.² In response, the VHA created the group practice manager (GPM), a new position responsible for improving clinical practice management and unifying access improvement across leadership levels, professions, and services within each local medical system.

In May 2015, the VHA began hiring and training GPMs to spearhead management of access to services. The US Department of Veterans Affairs (VA) Office of Veteran Access to Care spearheaded GPM training, including face-to-face sessions, national calls, webinars, and educational materials. Five local medical systems were selected by the VA Office of Veteran Access to Care to implement the GPM role to allow for an early evaluation of the program that would inform a subsequent nationwide rollout. Implementation of the GPM role remained in the hands of local medical systems.

Longer wait times are shown to impact patient health.^3,4 Open access scheduling and other patient-centered access management interventions have been shown to improve availability of primary care appointments.⁵ However, little empirical evidence exists regarding the managers who focus on clinic access interventions. While the nonpeer-reviewed literature includes references to such roles, including GPMs, the empirical literature has focused on external practice faciliators,^6-8 “mid-level managers,”⁹ and clinic staff.¹⁰ We found no peer-reviewed articles on the needs and experiences of practice managers who are focused on improving access. The purpose of this study was to examine GPM prototype sites to both enhance subsequent nationwide implementation and to advance empirical literature on managing patient access within health care.

Methods

In 2015, the VA identified 5 prototype sites representing diverse geographic locations, size, and complexity for the implementation of the GPM role (Table 1). These sites self-identified as having clinical practice management experience. GPMs attended 4 training sessions between February and August 2015.

Data Collection

Participants from each prototype site included GPMs, national trainers, clinic leaders, and frontline staff. Table 2 includes the roles and sample size. Participants were recruited through purposive sampling followed by snowball sampling until thematic saturation was reached (the point at which subsequent data fail to produce new findings across sites and roles of interest).

Guided by the Consolidated Framework for Implementation Research (CFIR), the research team developed semistructured interview guides tailored to participants’ roles to elicit rich descriptions regarding overall impressions, practice management strategies, goals, activities, relationship to clinic roles, data analytics usage, challenges, barriers, and facilitators.¹¹ These guides included open-ended questions and structured prompts utilizing participant language for follow-up probes to minimize interviewing bias (eAppendix:

Data Analysis

Data were analyzed using iterative deductive and inductive content analysis.¹² Deductive content analysis consisted of identifying quotes that fit within preidentified categories (ie, perceptions of national effort, organizational structure for GPM, challenges, facilitators, metrics and tools, and mobilizing access culture) developed by the interdisciplinary research team. Further content analysis entailed open-coding and iteratively revisiting and reconciling codes associated within each preidentified category as new codes emerged. The team analyzed the resulting codes to inductively and iteratively identify and stabilize themes regarding the GPM role: roles and tasks, GPM characteristics, issues, and challenges. Through this process we moved coded data to reconciled descriptions suited to addressing the purposes of this study. Dedoose 7.0.23 software was used for qualitative data management and analysis.

Results

The study identified participants’ overall impressions of the GPM initiative and key themes within 4 major domains regarding implementing the GPM role: roles and tasks (implementing clinic practice management, leading patient access, supporting data analytics, and enabling self and staff); GPM characteristics (familiarity with clinical services, knowledge of VHA systems, ability to analyze patient data, communication skills, and the ability to work with others); and issues, and challenges (technical, social, and structural).

Overall Impressions

Interviewees perceived the GPM initiative as a consolidation of existing distributed responsibilities into one role that directly reported to local top-level management with indirect reporting to national leaders. Many of the sites reported that they had designated or planned to designate a role resembling the GPM prior to the initiative. “There are staff who’ve been doing some of this work all along,” a GPM noted. “We just didn’t have them grouped together. They weren’t necessarily all working in the same type of service under the same type of structure.”

Whether the GPM position was new or not, participants referenced the importance and challenges of engaging the local facility in recognizing the agency associated with the GPM position. According to national support, the staff are trying to get the facility to understand “why the group practice manager is so important… we’ve got to embed that standard position in the system.”

While the GPM was recognized as the hub of access management, respondents recognized that transformation regarding access involved many players. “We have to create [an] orchestrated team inside each facility,” an advisor argued.

Respondents discussed how the initiative allows local facilities to appoint a specific person with a specific title and role who helps facilitate, organize, and legitimize an access focus at their sites. One GPM interviewee noted how the initiative helped refocus some of their previously less centralized efforts. “We’ve always looked at productivity; we’ve always looked at access; we’ve always looked at efficiency. I think the bigger difference is now there are individuals identified in the clinics, as practice managers as well…I interact with them. They interact with individual clinic staff, and it’s more of a group process than a single individual.”

The value of having tools available and being able to track and manage patient care as a specific example of the positive impact of this new role was noted by participants. A GPM noted that many health care providers will be happy to have tools to better manage their services and a process “that flows from a service level all the way up to executive management, where there is a common interest in making those things happen—I think that’s going to be a tremendous help.”

Participants expressed concern that the national GPM rollout would be a one-size-fits-all approach. These respondents emphasized the need to have the flexibility to customize their activities to meet their unique site and patient needs.

GPM Roles and Tasks

Participants described 4 primary roles that the GPM was expected to fill: implementing clinic practice management, leading patient access, supporting data analytics, and enabling self and staff. Some activities overlapped in that they served to support multiple role areas (Figure 1).

Implementing clinic practice management. In the early stages of the initiative, the GPM’s primary role was to prepare the facility to implement a standardized set of clinic practice management (CPM) team processes. Part of standardizing the CPM process was defining the scope and tasks of the GPM, which requires significant planning for the implementation. “My big job is to finalize what we think group practice management is going to look [like] here,” a GPM reported.

Each prototype site had latitude to interpret the GPM initiative in a way that would work in their context within given VHA boundaries and ongoing initiatives. To achieve the high-level vision and purpose, the GPM first had to develop action plans that accounted for the operating environment of the facility. According to one GPM, VA national officials are “constantly” asking for action plans, which required significant time by specific deadlines. “They want an action plan [and to] clean up all your consults, [and to] clean up all your recall reminders.”

Leading on improving access efforts. Participants saw the GPM as the central staff member responsible for providing oversight of any activities and people involved in improving access. “I ensure everybody is doing what they’re supposed to do,” one GPM reported. When the GPM sees areas that are not being addressed, the individual tries to develop a process or training to “close those gaps.”

GPMs promoted an awareness of their goals, changes in process, and new tools accompanying the initiative. However, other access initiatives were occurring simultaneously creating confusion for health care providers and patients; thus GPMs found they were managing a wide array of related initiatives.

GPMs have to negotiate with leaders across the VHA facility, many of whom operated at a higher leadership level and had different priorities, to address access problems.

“I’m a lieutenant as a GPM in a clinic, a GPM noted. “How is the lieutenant going to talk to a major or a colonel in the clinic and say your clinic has problems. How[‘s] that lieutenant...going to do that? With people skills!”

Managing expectations about the speed and to what extent a problem could be resolved was an important part of the GPM leadership role. “I see myself as managing expectations both up to the leadership and down to the frontline,” a GPM explained. “I find myself talking to leadership [about] our progress. But at the same time, we have to say, ‘not everything can be fixed overnight.’”

Providing leadership on access-related issues included developing a range of options for addressing patient access problems. One analytics manager recounted how the GPM role led to evaluating how physical space limited efficiency in clinic flow. The first step was identifying possible additional rooms to improve clinic flow. This required working with the space committee to “get someone to look at our overarching space and find someplace else for them to sit” to avoid adding to congestion in the clinic area.
 

Supporting data analytics. Given the importance of data analytics, GPMs had a critical role in helping to ensure that the data were accurate and clean. At one facility the GPM and the business managers, “are doing a tremendous amount to clean up our data to make it accurately reflect what it is that we’re doing,” reported a community clinic director.

GPMs routinely immersed themselves in the data to understand access issues. GPMs worked with clinic leaders to identify the underlying causes and various solutions. The GPMs maintained access through administrative scrubbing of the data and finding “smart ways to get patients scheduled,” a GPM explained. “I don’t think our facility would have taken care of as many veterans in the time frame as we did....We’ve cleared over 4,000 consults that were older than 90 days. We have cleared thousands and thousands of weekly reminders.”

GPMs expressed the need for aggregated (ie, dashboard) and standardized information to efficiently address access issues. “I would like to have some more standardization on what’s being reviewed; it seems to change frequently, and so [to] be able to track and trend and have something given to me to review,” one health care provider requested. On the other hand, participants also described a need for decision support tools that would lead to action aligned with best practices. “Instead of a dashboard or something that’s just measuring their performance, it’s more something that they can look at and take action” a national support staff advisor suggested.

Enabling self and staff. GPMs felt they were most effective if they enabled themselves and stakeholders through training and by cultivating relationships and team building. Figure 2 illustrates the various stakeholders GPMs reported engaging with. The GPMS should be building relationships, bridging relationships, developing trust, and then providing a higher level of hands-on management. However, “that doesn’t really happen right now, day to day,” one member of leadership reported.

Key topics in GPM leadership training included both soft skills (change management, culture change, and negotiation skills) and crucial analytic/technical training (understanding each metric and dashboard available, data analytics, and supply/demand balancing techniques). The GPMs not only wanted to understand metrics as part of their training, but also want to know what to do about them.

An “operationalization” training approach (discerning the meaning of data, data-based decision making, and determining action from multiple options) inspired by real-life situations was preferred by participants. Other effective learning structures included job aids in the form of templated Gantt charts, process maps to guide GPMs through implementation of new processes, formalized peer learning (accumulated field insights shared during training courses), and informal peer sharing of direct experiences during calls.

GPMs also emphasized training for frontline clinical and support staff, including schedulers. VHA schedulers typically have less education and experience higher turnover rates than do other clinic staff, yet they carry out complex and critical tasks. Providing training and ensuring that any materials developed for training and education were appropriate to the level of education of schedulers was an important task for GPMs. “If they don’t understand all of the scheduling principles and potential,” one GPM explained, “we will not be maximizing the utilization of our parts.”

GPMs also provided informal education to clinicians. Participants noted GPMs have to avoid appearing to overstep their positions or presuming more knowledge and expertise than clinicians. They “have to be able to teach a physician without being overbearing, in a way a physician will accept it as advisement,” one program leader reported.
 

GPM knowledge, skills, and abilities. GPMs presented a complex range of knowledge, skills, and abilities, including clinical, administrative, analytics, and people skills. All interviewees reported that their prior education and experience did not sufficiently train them for the GPM role. GPMs identified a willingness to learn quickly as a critical characteristic. Many GPMs tended to have a formal education in health administration or business (eg, MBAs); others had administrative experience (eg, administrative assistance to executive leadership) or clinical training (eg, physician assistant). Detailed clinical knowledge was not expected, but clinical familiarity was helpful.

Some interviewees also mentioned previous experience and familiarity with the VHA system specifically as an advantage. This was especially true for VA outpatient flows, clinic flows, and understanding what an outpatient is in a VA context. Interviewees noted the importance of GPMs needing to be able to analyze patient demand metrics and underlying data in order to determine supply of providers and then to allocate adequate resources to complement providers. Forecasting skills were referenced as a key point. “They need to be able to be assured that they can recruit more providers if needed,” a national support staff advisor noted.

Given the importance of developing effective relationships, communication skills were mentioned by most participants and underscored as critical to establishing trust between GPMs and others as the initiative was being implemented. Interviewees indicated that relationship building was further enhanced when GPMs possessed the ability to “work with” rather than command clinicians and staff; navigate politics; and were respectful of other people’s knowledge, skills, abilities, and status. “They have to work with the nursing staff and teach them,” a leader described, “so that people understand that we are going to a different place to achieve our primary objectives and goals.”

Issues and Challenges

Participants identified several technical, social, and structural challenges and barriers to successfully implementing the GPM role.

Technical challenges. Recurring themes across all phases of data analytics were GPMs’ capability to challenge data use and use large volumes of information from multiple data sources (entering and accessing data; “drilling down” from summaries; generating reports; and analyzing and interpreting resulting metrics). Interviewees reported that information assessment and analytic support were not consistent. One GPM had a data analyst pulling reports needed to support clinical units while other GPMs trained staff to pull data. Even with support, some GPMs had issues due to limited information technology (IT) skills or access privileges leading to inefficiencies and delays. “Whenever I need anything from a programmer, I have to go through, you know, the IT gods in the sky,” one GPM remarked. “That usually takes a few months or more.”

Social challenges. Instituting the GPM role was a cultural change, and interviewees reported needing to address resistance to CPM model efforts. Resistance to change “is particularly hard in the VA just because it has a unique culture,” one leader noted. “There is a comfort in the legacy way of doing things.” The GPM initiative was introduced during a time when other national level initiatives were being implemented throughout the VHA. Fulfilling requests for information for these initiatives became the responsibility of the GPM and their team, which diverted attention from the mandate to improve access. Furthermore, GPMs were often considered the “change communicators” to clinics putting them in the role of “bad messenger,” which degraded trust and made it difficult to partner with clinicians.

Efforts to work through change management and build relationships included general program awareness presentation to internal stakeholders; including key stakeholders in GPM committees; pre-emptive conversations with unit chiefs; creating awareness of the GPM activities and progress through formal and informal update meetings; and identifying successes regarding access.

Structural challenges. The GPM role did not have direct supervision over clinical and administrative leaders, making it challenging to enact change. GPMs reported that “they do not always have authority over the area that they are being asked to manage,” which made their work difficult, requiring strong negotiation skills and political savvy to affect change. However, as the clinic staff and providers saw how the GPM could support and positively impact their practice, these challenges began to subside.

Discussion

This study provides empirical evidence regarding the implementation of a new access management strategy for health care systems focused on improving timeliness of care. First, the GPM position was seen as critical at each facility, as a single point person, to help local system leaders respond effectively to both national mandates and local context. Second, requiring the GPMs to report to the medical center director or chief of staff was important for integrating access perspectives across service lines and to facilitate a strong GPM role in strategic planning. Third, the intentional flexibility of the access management initiative, beyond the nationally specified aspects of the GPM role, was key for allowing individual sites to adapt to unique local challenges, resources, and population demands. Fourth, the initiative provided GPMs with opportunities to learn important skills and support the acquisition, utilization, and communication of a tremendous range of data toward responsive action.

According to our respondents, the GPM role demands functioning across a broad set of responsibilities; understanding the big picture as well as the complex underlying variables; engaging facility leaders, clinical and administrative staff; and prioritizing competing national and local demands. Consistent with previous findings, effective GPMs must possess a complex set of skills (interpersonal, analytic, and leadership) and the ability to create a supporting team.¹³

In practice, improving access at individual sites of care (VA medical centers and community-based outpatient clinics) poses significant challenges, especially in the early stages, even with the assistance of a GPM. For example, some respondents reported being overwhelmed by the volume of available data and dashboards, and responding to current requests for data analysis and dissemination sometimes impeded long range planning. Multiple national access-related initiatives and local pressures also generated excessive and potentially conflicting demands. Thus, while the creation of a GPM position seemed to be essential for the pilot sites to improve local access and timeliness to care, success also requires ongoing national and facility-wide communication, education, and support. Ongoing data analysis training and support will be critical to ensuring the sustainability of the position. Last, recruiting GPMs with the needed complex skill set may prove to be challenging, and it will be important to provide resources to identify, attract, and retain well-qualified GPMs.

Limitations and Future Work

This study was based on a small initial sample of pilot sites of varying sizes and, as such, may not reflect the experience of all VHA GPMs. In addition, the use of snowball sampling, while facilitating identification of key stakeholders, may introduce bias in participant sampling. Nonetheless, the results from this study provide findings that informed the national VHA GPM initiative and can inform further studies of practice management roles outside of the VA.

Further study of the VHA GPM implementation and similar roles in other health care systems is needed. As the GPM position is fully implemented across the VHA, large scale evaluation is needed to gain a more representative picture and allow for comparison of the GPM role at various types of facilities (eg, size, rurality, complexity, ranking based on access performance metrics).

Conclusion

Improving access to care is a central goal for health care systems. The incorporation of the GPM role is an innovative approach to improve access management strategies. Early study of prototype sites provided VHA leadership with valuable insights used to influence further rollout of this initiative. Based on our findings, national and local support are important to ongoing success. National access mandates, training, and resources should focus on ensuring sufficient GPM authority, enabling GPMs to use data, and ensuring GPMs engage with frontline clinical and administrative staff to improve veteran access to care.

The Veterans Health Administration (VHA) provides care for 9 million veterans at 1,255 health care sites linked to one of 170 local medical systems.¹ Recognizing that providing timely care requires effective access management, the US Congress mandated training of VHA staff to manage and improve access to care but did not provide additional local funds for new positions.² In response, the VHA created the group practice manager (GPM), a new position responsible for improving clinical practice management and unifying access improvement across leadership levels, professions, and services within each local medical system.

In May 2015, the VHA began hiring and training GPMs to spearhead management of access to services. The US Department of Veterans Affairs (VA) Office of Veteran Access to Care spearheaded GPM training, including face-to-face sessions, national calls, webinars, and educational materials. Five local medical systems were selected by the VA Office of Veteran Access to Care to implement the GPM role to allow for an early evaluation of the program that would inform a subsequent nationwide rollout. Implementation of the GPM role remained in the hands of local medical systems.

Longer wait times are shown to impact patient health.^3,4 Open access scheduling and other patient-centered access management interventions have been shown to improve availability of primary care appointments.⁵ However, little empirical evidence exists regarding the managers who focus on clinic access interventions. While the nonpeer-reviewed literature includes references to such roles, including GPMs, the empirical literature has focused on external practice faciliators,^6-8 “mid-level managers,”⁹ and clinic staff.¹⁰ We found no peer-reviewed articles on the needs and experiences of practice managers who are focused on improving access. The purpose of this study was to examine GPM prototype sites to both enhance subsequent nationwide implementation and to advance empirical literature on managing patient access within health care.

Methods

In 2015, the VA identified 5 prototype sites representing diverse geographic locations, size, and complexity for the implementation of the GPM role (Table 1). These sites self-identified as having clinical practice management experience. GPMs attended 4 training sessions between February and August 2015.

Data Collection

Participants from each prototype site included GPMs, national trainers, clinic leaders, and frontline staff. Table 2 includes the roles and sample size. Participants were recruited through purposive sampling followed by snowball sampling until thematic saturation was reached (the point at which subsequent data fail to produce new findings across sites and roles of interest).

Guided by the Consolidated Framework for Implementation Research (CFIR), the research team developed semistructured interview guides tailored to participants’ roles to elicit rich descriptions regarding overall impressions, practice management strategies, goals, activities, relationship to clinic roles, data analytics usage, challenges, barriers, and facilitators.¹¹ These guides included open-ended questions and structured prompts utilizing participant language for follow-up probes to minimize interviewing bias (eAppendix:

Data Analysis

Data were analyzed using iterative deductive and inductive content analysis.¹² Deductive content analysis consisted of identifying quotes that fit within preidentified categories (ie, perceptions of national effort, organizational structure for GPM, challenges, facilitators, metrics and tools, and mobilizing access culture) developed by the interdisciplinary research team. Further content analysis entailed open-coding and iteratively revisiting and reconciling codes associated within each preidentified category as new codes emerged. The team analyzed the resulting codes to inductively and iteratively identify and stabilize themes regarding the GPM role: roles and tasks, GPM characteristics, issues, and challenges. Through this process we moved coded data to reconciled descriptions suited to addressing the purposes of this study. Dedoose 7.0.23 software was used for qualitative data management and analysis.

Results

The study identified participants’ overall impressions of the GPM initiative and key themes within 4 major domains regarding implementing the GPM role: roles and tasks (implementing clinic practice management, leading patient access, supporting data analytics, and enabling self and staff); GPM characteristics (familiarity with clinical services, knowledge of VHA systems, ability to analyze patient data, communication skills, and the ability to work with others); and issues, and challenges (technical, social, and structural).

Overall Impressions

Interviewees perceived the GPM initiative as a consolidation of existing distributed responsibilities into one role that directly reported to local top-level management with indirect reporting to national leaders. Many of the sites reported that they had designated or planned to designate a role resembling the GPM prior to the initiative. “There are staff who’ve been doing some of this work all along,” a GPM noted. “We just didn’t have them grouped together. They weren’t necessarily all working in the same type of service under the same type of structure.”

Whether the GPM position was new or not, participants referenced the importance and challenges of engaging the local facility in recognizing the agency associated with the GPM position. According to national support, the staff are trying to get the facility to understand “why the group practice manager is so important… we’ve got to embed that standard position in the system.”

While the GPM was recognized as the hub of access management, respondents recognized that transformation regarding access involved many players. “We have to create [an] orchestrated team inside each facility,” an advisor argued.

Respondents discussed how the initiative allows local facilities to appoint a specific person with a specific title and role who helps facilitate, organize, and legitimize an access focus at their sites. One GPM interviewee noted how the initiative helped refocus some of their previously less centralized efforts. “We’ve always looked at productivity; we’ve always looked at access; we’ve always looked at efficiency. I think the bigger difference is now there are individuals identified in the clinics, as practice managers as well…I interact with them. They interact with individual clinic staff, and it’s more of a group process than a single individual.”

The value of having tools available and being able to track and manage patient care as a specific example of the positive impact of this new role was noted by participants. A GPM noted that many health care providers will be happy to have tools to better manage their services and a process “that flows from a service level all the way up to executive management, where there is a common interest in making those things happen—I think that’s going to be a tremendous help.”

Participants expressed concern that the national GPM rollout would be a one-size-fits-all approach. These respondents emphasized the need to have the flexibility to customize their activities to meet their unique site and patient needs.

GPM Roles and Tasks

Participants described 4 primary roles that the GPM was expected to fill: implementing clinic practice management, leading patient access, supporting data analytics, and enabling self and staff. Some activities overlapped in that they served to support multiple role areas (Figure 1).

Implementing clinic practice management. In the early stages of the initiative, the GPM’s primary role was to prepare the facility to implement a standardized set of clinic practice management (CPM) team processes. Part of standardizing the CPM process was defining the scope and tasks of the GPM, which requires significant planning for the implementation. “My big job is to finalize what we think group practice management is going to look [like] here,” a GPM reported.

Each prototype site had latitude to interpret the GPM initiative in a way that would work in their context within given VHA boundaries and ongoing initiatives. To achieve the high-level vision and purpose, the GPM first had to develop action plans that accounted for the operating environment of the facility. According to one GPM, VA national officials are “constantly” asking for action plans, which required significant time by specific deadlines. “They want an action plan [and to] clean up all your consults, [and to] clean up all your recall reminders.”

Leading on improving access efforts. Participants saw the GPM as the central staff member responsible for providing oversight of any activities and people involved in improving access. “I ensure everybody is doing what they’re supposed to do,” one GPM reported. When the GPM sees areas that are not being addressed, the individual tries to develop a process or training to “close those gaps.”

GPMs promoted an awareness of their goals, changes in process, and new tools accompanying the initiative. However, other access initiatives were occurring simultaneously creating confusion for health care providers and patients; thus GPMs found they were managing a wide array of related initiatives.

GPMs have to negotiate with leaders across the VHA facility, many of whom operated at a higher leadership level and had different priorities, to address access problems.

“I’m a lieutenant as a GPM in a clinic, a GPM noted. “How is the lieutenant going to talk to a major or a colonel in the clinic and say your clinic has problems. How[‘s] that lieutenant...going to do that? With people skills!”

Managing expectations about the speed and to what extent a problem could be resolved was an important part of the GPM leadership role. “I see myself as managing expectations both up to the leadership and down to the frontline,” a GPM explained. “I find myself talking to leadership [about] our progress. But at the same time, we have to say, ‘not everything can be fixed overnight.’”

Providing leadership on access-related issues included developing a range of options for addressing patient access problems. One analytics manager recounted how the GPM role led to evaluating how physical space limited efficiency in clinic flow. The first step was identifying possible additional rooms to improve clinic flow. This required working with the space committee to “get someone to look at our overarching space and find someplace else for them to sit” to avoid adding to congestion in the clinic area.
 

Supporting data analytics. Given the importance of data analytics, GPMs had a critical role in helping to ensure that the data were accurate and clean. At one facility the GPM and the business managers, “are doing a tremendous amount to clean up our data to make it accurately reflect what it is that we’re doing,” reported a community clinic director.

GPMs routinely immersed themselves in the data to understand access issues. GPMs worked with clinic leaders to identify the underlying causes and various solutions. The GPMs maintained access through administrative scrubbing of the data and finding “smart ways to get patients scheduled,” a GPM explained. “I don’t think our facility would have taken care of as many veterans in the time frame as we did....We’ve cleared over 4,000 consults that were older than 90 days. We have cleared thousands and thousands of weekly reminders.”

GPMs expressed the need for aggregated (ie, dashboard) and standardized information to efficiently address access issues. “I would like to have some more standardization on what’s being reviewed; it seems to change frequently, and so [to] be able to track and trend and have something given to me to review,” one health care provider requested. On the other hand, participants also described a need for decision support tools that would lead to action aligned with best practices. “Instead of a dashboard or something that’s just measuring their performance, it’s more something that they can look at and take action” a national support staff advisor suggested.

Enabling self and staff. GPMs felt they were most effective if they enabled themselves and stakeholders through training and by cultivating relationships and team building. Figure 2 illustrates the various stakeholders GPMs reported engaging with. The GPMS should be building relationships, bridging relationships, developing trust, and then providing a higher level of hands-on management. However, “that doesn’t really happen right now, day to day,” one member of leadership reported.

Key topics in GPM leadership training included both soft skills (change management, culture change, and negotiation skills) and crucial analytic/technical training (understanding each metric and dashboard available, data analytics, and supply/demand balancing techniques). The GPMs not only wanted to understand metrics as part of their training, but also want to know what to do about them.

An “operationalization” training approach (discerning the meaning of data, data-based decision making, and determining action from multiple options) inspired by real-life situations was preferred by participants. Other effective learning structures included job aids in the form of templated Gantt charts, process maps to guide GPMs through implementation of new processes, formalized peer learning (accumulated field insights shared during training courses), and informal peer sharing of direct experiences during calls.

GPMs also emphasized training for frontline clinical and support staff, including schedulers. VHA schedulers typically have less education and experience higher turnover rates than do other clinic staff, yet they carry out complex and critical tasks. Providing training and ensuring that any materials developed for training and education were appropriate to the level of education of schedulers was an important task for GPMs. “If they don’t understand all of the scheduling principles and potential,” one GPM explained, “we will not be maximizing the utilization of our parts.”

GPMs also provided informal education to clinicians. Participants noted GPMs have to avoid appearing to overstep their positions or presuming more knowledge and expertise than clinicians. They “have to be able to teach a physician without being overbearing, in a way a physician will accept it as advisement,” one program leader reported.
 

GPM knowledge, skills, and abilities. GPMs presented a complex range of knowledge, skills, and abilities, including clinical, administrative, analytics, and people skills. All interviewees reported that their prior education and experience did not sufficiently train them for the GPM role. GPMs identified a willingness to learn quickly as a critical characteristic. Many GPMs tended to have a formal education in health administration or business (eg, MBAs); others had administrative experience (eg, administrative assistance to executive leadership) or clinical training (eg, physician assistant). Detailed clinical knowledge was not expected, but clinical familiarity was helpful.

Some interviewees also mentioned previous experience and familiarity with the VHA system specifically as an advantage. This was especially true for VA outpatient flows, clinic flows, and understanding what an outpatient is in a VA context. Interviewees noted the importance of GPMs needing to be able to analyze patient demand metrics and underlying data in order to determine supply of providers and then to allocate adequate resources to complement providers. Forecasting skills were referenced as a key point. “They need to be able to be assured that they can recruit more providers if needed,” a national support staff advisor noted.

Given the importance of developing effective relationships, communication skills were mentioned by most participants and underscored as critical to establishing trust between GPMs and others as the initiative was being implemented. Interviewees indicated that relationship building was further enhanced when GPMs possessed the ability to “work with” rather than command clinicians and staff; navigate politics; and were respectful of other people’s knowledge, skills, abilities, and status. “They have to work with the nursing staff and teach them,” a leader described, “so that people understand that we are going to a different place to achieve our primary objectives and goals.”

Issues and Challenges

Participants identified several technical, social, and structural challenges and barriers to successfully implementing the GPM role.

Technical challenges. Recurring themes across all phases of data analytics were GPMs’ capability to challenge data use and use large volumes of information from multiple data sources (entering and accessing data; “drilling down” from summaries; generating reports; and analyzing and interpreting resulting metrics). Interviewees reported that information assessment and analytic support were not consistent. One GPM had a data analyst pulling reports needed to support clinical units while other GPMs trained staff to pull data. Even with support, some GPMs had issues due to limited information technology (IT) skills or access privileges leading to inefficiencies and delays. “Whenever I need anything from a programmer, I have to go through, you know, the IT gods in the sky,” one GPM remarked. “That usually takes a few months or more.”

Social challenges. Instituting the GPM role was a cultural change, and interviewees reported needing to address resistance to CPM model efforts. Resistance to change “is particularly hard in the VA just because it has a unique culture,” one leader noted. “There is a comfort in the legacy way of doing things.” The GPM initiative was introduced during a time when other national level initiatives were being implemented throughout the VHA. Fulfilling requests for information for these initiatives became the responsibility of the GPM and their team, which diverted attention from the mandate to improve access. Furthermore, GPMs were often considered the “change communicators” to clinics putting them in the role of “bad messenger,” which degraded trust and made it difficult to partner with clinicians.

Efforts to work through change management and build relationships included general program awareness presentation to internal stakeholders; including key stakeholders in GPM committees; pre-emptive conversations with unit chiefs; creating awareness of the GPM activities and progress through formal and informal update meetings; and identifying successes regarding access.

Structural challenges. The GPM role did not have direct supervision over clinical and administrative leaders, making it challenging to enact change. GPMs reported that “they do not always have authority over the area that they are being asked to manage,” which made their work difficult, requiring strong negotiation skills and political savvy to affect change. However, as the clinic staff and providers saw how the GPM could support and positively impact their practice, these challenges began to subside.

Discussion

This study provides empirical evidence regarding the implementation of a new access management strategy for health care systems focused on improving timeliness of care. First, the GPM position was seen as critical at each facility, as a single point person, to help local system leaders respond effectively to both national mandates and local context. Second, requiring the GPMs to report to the medical center director or chief of staff was important for integrating access perspectives across service lines and to facilitate a strong GPM role in strategic planning. Third, the intentional flexibility of the access management initiative, beyond the nationally specified aspects of the GPM role, was key for allowing individual sites to adapt to unique local challenges, resources, and population demands. Fourth, the initiative provided GPMs with opportunities to learn important skills and support the acquisition, utilization, and communication of a tremendous range of data toward responsive action.

According to our respondents, the GPM role demands functioning across a broad set of responsibilities; understanding the big picture as well as the complex underlying variables; engaging facility leaders, clinical and administrative staff; and prioritizing competing national and local demands. Consistent with previous findings, effective GPMs must possess a complex set of skills (interpersonal, analytic, and leadership) and the ability to create a supporting team.¹³

In practice, improving access at individual sites of care (VA medical centers and community-based outpatient clinics) poses significant challenges, especially in the early stages, even with the assistance of a GPM. For example, some respondents reported being overwhelmed by the volume of available data and dashboards, and responding to current requests for data analysis and dissemination sometimes impeded long range planning. Multiple national access-related initiatives and local pressures also generated excessive and potentially conflicting demands. Thus, while the creation of a GPM position seemed to be essential for the pilot sites to improve local access and timeliness to care, success also requires ongoing national and facility-wide communication, education, and support. Ongoing data analysis training and support will be critical to ensuring the sustainability of the position. Last, recruiting GPMs with the needed complex skill set may prove to be challenging, and it will be important to provide resources to identify, attract, and retain well-qualified GPMs.

Limitations and Future Work

This study was based on a small initial sample of pilot sites of varying sizes and, as such, may not reflect the experience of all VHA GPMs. In addition, the use of snowball sampling, while facilitating identification of key stakeholders, may introduce bias in participant sampling. Nonetheless, the results from this study provide findings that informed the national VHA GPM initiative and can inform further studies of practice management roles outside of the VA.

Further study of the VHA GPM implementation and similar roles in other health care systems is needed. As the GPM position is fully implemented across the VHA, large scale evaluation is needed to gain a more representative picture and allow for comparison of the GPM role at various types of facilities (eg, size, rurality, complexity, ranking based on access performance metrics).

Conclusion

Improving access to care is a central goal for health care systems. The incorporation of the GPM role is an innovative approach to improve access management strategies. Early study of prototype sites provided VHA leadership with valuable insights used to influence further rollout of this initiative. Based on our findings, national and local support are important to ongoing success. National access mandates, training, and resources should focus on ensuring sufficient GPM authority, enabling GPMs to use data, and ensuring GPMs engage with frontline clinical and administrative staff to improve veteran access to care.

The Veterans Health Administration (VHA) provides care for 9 million veterans at 1,255 health care sites linked to one of 170 local medical systems.¹ Recognizing that providing timely care requires effective access management, the US Congress mandated training of VHA staff to manage and improve access to care but did not provide additional local funds for new positions.² In response, the VHA created the group practice manager (GPM), a new position responsible for improving clinical practice management and unifying access improvement across leadership levels, professions, and services within each local medical system.

In May 2015, the VHA began hiring and training GPMs to spearhead management of access to services. The US Department of Veterans Affairs (VA) Office of Veteran Access to Care spearheaded GPM training, including face-to-face sessions, national calls, webinars, and educational materials. Five local medical systems were selected by the VA Office of Veteran Access to Care to implement the GPM role to allow for an early evaluation of the program that would inform a subsequent nationwide rollout. Implementation of the GPM role remained in the hands of local medical systems.

Longer wait times are shown to impact patient health.^3,4 Open access scheduling and other patient-centered access management interventions have been shown to improve availability of primary care appointments.⁵ However, little empirical evidence exists regarding the managers who focus on clinic access interventions. While the nonpeer-reviewed literature includes references to such roles, including GPMs, the empirical literature has focused on external practice faciliators,^6-8 “mid-level managers,”⁹ and clinic staff.¹⁰ We found no peer-reviewed articles on the needs and experiences of practice managers who are focused on improving access. The purpose of this study was to examine GPM prototype sites to both enhance subsequent nationwide implementation and to advance empirical literature on managing patient access within health care.

Methods

In 2015, the VA identified 5 prototype sites representing diverse geographic locations, size, and complexity for the implementation of the GPM role (Table 1). These sites self-identified as having clinical practice management experience. GPMs attended 4 training sessions between February and August 2015.

Data Collection

Participants from each prototype site included GPMs, national trainers, clinic leaders, and frontline staff. Table 2 includes the roles and sample size. Participants were recruited through purposive sampling followed by snowball sampling until thematic saturation was reached (the point at which subsequent data fail to produce new findings across sites and roles of interest).

Guided by the Consolidated Framework for Implementation Research (CFIR), the research team developed semistructured interview guides tailored to participants’ roles to elicit rich descriptions regarding overall impressions, practice management strategies, goals, activities, relationship to clinic roles, data analytics usage, challenges, barriers, and facilitators.¹¹ These guides included open-ended questions and structured prompts utilizing participant language for follow-up probes to minimize interviewing bias (eAppendix:

Data Analysis

Data were analyzed using iterative deductive and inductive content analysis.¹² Deductive content analysis consisted of identifying quotes that fit within preidentified categories (ie, perceptions of national effort, organizational structure for GPM, challenges, facilitators, metrics and tools, and mobilizing access culture) developed by the interdisciplinary research team. Further content analysis entailed open-coding and iteratively revisiting and reconciling codes associated within each preidentified category as new codes emerged. The team analyzed the resulting codes to inductively and iteratively identify and stabilize themes regarding the GPM role: roles and tasks, GPM characteristics, issues, and challenges. Through this process we moved coded data to reconciled descriptions suited to addressing the purposes of this study. Dedoose 7.0.23 software was used for qualitative data management and analysis.

Results

The study identified participants’ overall impressions of the GPM initiative and key themes within 4 major domains regarding implementing the GPM role: roles and tasks (implementing clinic practice management, leading patient access, supporting data analytics, and enabling self and staff); GPM characteristics (familiarity with clinical services, knowledge of VHA systems, ability to analyze patient data, communication skills, and the ability to work with others); and issues, and challenges (technical, social, and structural).

Overall Impressions

Interviewees perceived the GPM initiative as a consolidation of existing distributed responsibilities into one role that directly reported to local top-level management with indirect reporting to national leaders. Many of the sites reported that they had designated or planned to designate a role resembling the GPM prior to the initiative. “There are staff who’ve been doing some of this work all along,” a GPM noted. “We just didn’t have them grouped together. They weren’t necessarily all working in the same type of service under the same type of structure.”

Whether the GPM position was new or not, participants referenced the importance and challenges of engaging the local facility in recognizing the agency associated with the GPM position. According to national support, the staff are trying to get the facility to understand “why the group practice manager is so important… we’ve got to embed that standard position in the system.”

While the GPM was recognized as the hub of access management, respondents recognized that transformation regarding access involved many players. “We have to create [an] orchestrated team inside each facility,” an advisor argued.

Respondents discussed how the initiative allows local facilities to appoint a specific person with a specific title and role who helps facilitate, organize, and legitimize an access focus at their sites. One GPM interviewee noted how the initiative helped refocus some of their previously less centralized efforts. “We’ve always looked at productivity; we’ve always looked at access; we’ve always looked at efficiency. I think the bigger difference is now there are individuals identified in the clinics, as practice managers as well…I interact with them. They interact with individual clinic staff, and it’s more of a group process than a single individual.”

The value of having tools available and being able to track and manage patient care as a specific example of the positive impact of this new role was noted by participants. A GPM noted that many health care providers will be happy to have tools to better manage their services and a process “that flows from a service level all the way up to executive management, where there is a common interest in making those things happen—I think that’s going to be a tremendous help.”

Participants expressed concern that the national GPM rollout would be a one-size-fits-all approach. These respondents emphasized the need to have the flexibility to customize their activities to meet their unique site and patient needs.

GPM Roles and Tasks

Participants described 4 primary roles that the GPM was expected to fill: implementing clinic practice management, leading patient access, supporting data analytics, and enabling self and staff. Some activities overlapped in that they served to support multiple role areas (Figure 1).

Implementing clinic practice management. In the early stages of the initiative, the GPM’s primary role was to prepare the facility to implement a standardized set of clinic practice management (CPM) team processes. Part of standardizing the CPM process was defining the scope and tasks of the GPM, which requires significant planning for the implementation. “My big job is to finalize what we think group practice management is going to look [like] here,” a GPM reported.

Each prototype site had latitude to interpret the GPM initiative in a way that would work in their context within given VHA boundaries and ongoing initiatives. To achieve the high-level vision and purpose, the GPM first had to develop action plans that accounted for the operating environment of the facility. According to one GPM, VA national officials are “constantly” asking for action plans, which required significant time by specific deadlines. “They want an action plan [and to] clean up all your consults, [and to] clean up all your recall reminders.”

Leading on improving access efforts. Participants saw the GPM as the central staff member responsible for providing oversight of any activities and people involved in improving access. “I ensure everybody is doing what they’re supposed to do,” one GPM reported. When the GPM sees areas that are not being addressed, the individual tries to develop a process or training to “close those gaps.”

GPMs promoted an awareness of their goals, changes in process, and new tools accompanying the initiative. However, other access initiatives were occurring simultaneously creating confusion for health care providers and patients; thus GPMs found they were managing a wide array of related initiatives.

GPMs have to negotiate with leaders across the VHA facility, many of whom operated at a higher leadership level and had different priorities, to address access problems.

“I’m a lieutenant as a GPM in a clinic, a GPM noted. “How is the lieutenant going to talk to a major or a colonel in the clinic and say your clinic has problems. How[‘s] that lieutenant...going to do that? With people skills!”

Managing expectations about the speed and to what extent a problem could be resolved was an important part of the GPM leadership role. “I see myself as managing expectations both up to the leadership and down to the frontline,” a GPM explained. “I find myself talking to leadership [about] our progress. But at the same time, we have to say, ‘not everything can be fixed overnight.’”

Providing leadership on access-related issues included developing a range of options for addressing patient access problems. One analytics manager recounted how the GPM role led to evaluating how physical space limited efficiency in clinic flow. The first step was identifying possible additional rooms to improve clinic flow. This required working with the space committee to “get someone to look at our overarching space and find someplace else for them to sit” to avoid adding to congestion in the clinic area.
 

Supporting data analytics. Given the importance of data analytics, GPMs had a critical role in helping to ensure that the data were accurate and clean. At one facility the GPM and the business managers, “are doing a tremendous amount to clean up our data to make it accurately reflect what it is that we’re doing,” reported a community clinic director.

GPMs routinely immersed themselves in the data to understand access issues. GPMs worked with clinic leaders to identify the underlying causes and various solutions. The GPMs maintained access through administrative scrubbing of the data and finding “smart ways to get patients scheduled,” a GPM explained. “I don’t think our facility would have taken care of as many veterans in the time frame as we did....We’ve cleared over 4,000 consults that were older than 90 days. We have cleared thousands and thousands of weekly reminders.”

GPMs expressed the need for aggregated (ie, dashboard) and standardized information to efficiently address access issues. “I would like to have some more standardization on what’s being reviewed; it seems to change frequently, and so [to] be able to track and trend and have something given to me to review,” one health care provider requested. On the other hand, participants also described a need for decision support tools that would lead to action aligned with best practices. “Instead of a dashboard or something that’s just measuring their performance, it’s more something that they can look at and take action” a national support staff advisor suggested.

Enabling self and staff. GPMs felt they were most effective if they enabled themselves and stakeholders through training and by cultivating relationships and team building. Figure 2 illustrates the various stakeholders GPMs reported engaging with. The GPMS should be building relationships, bridging relationships, developing trust, and then providing a higher level of hands-on management. However, “that doesn’t really happen right now, day to day,” one member of leadership reported.

Key topics in GPM leadership training included both soft skills (change management, culture change, and negotiation skills) and crucial analytic/technical training (understanding each metric and dashboard available, data analytics, and supply/demand balancing techniques). The GPMs not only wanted to understand metrics as part of their training, but also want to know what to do about them.

An “operationalization” training approach (discerning the meaning of data, data-based decision making, and determining action from multiple options) inspired by real-life situations was preferred by participants. Other effective learning structures included job aids in the form of templated Gantt charts, process maps to guide GPMs through implementation of new processes, formalized peer learning (accumulated field insights shared during training courses), and informal peer sharing of direct experiences during calls.

GPMs also emphasized training for frontline clinical and support staff, including schedulers. VHA schedulers typically have less education and experience higher turnover rates than do other clinic staff, yet they carry out complex and critical tasks. Providing training and ensuring that any materials developed for training and education were appropriate to the level of education of schedulers was an important task for GPMs. “If they don’t understand all of the scheduling principles and potential,” one GPM explained, “we will not be maximizing the utilization of our parts.”

GPMs also provided informal education to clinicians. Participants noted GPMs have to avoid appearing to overstep their positions or presuming more knowledge and expertise than clinicians. They “have to be able to teach a physician without being overbearing, in a way a physician will accept it as advisement,” one program leader reported.
 

GPM knowledge, skills, and abilities. GPMs presented a complex range of knowledge, skills, and abilities, including clinical, administrative, analytics, and people skills. All interviewees reported that their prior education and experience did not sufficiently train them for the GPM role. GPMs identified a willingness to learn quickly as a critical characteristic. Many GPMs tended to have a formal education in health administration or business (eg, MBAs); others had administrative experience (eg, administrative assistance to executive leadership) or clinical training (eg, physician assistant). Detailed clinical knowledge was not expected, but clinical familiarity was helpful.

Some interviewees also mentioned previous experience and familiarity with the VHA system specifically as an advantage. This was especially true for VA outpatient flows, clinic flows, and understanding what an outpatient is in a VA context. Interviewees noted the importance of GPMs needing to be able to analyze patient demand metrics and underlying data in order to determine supply of providers and then to allocate adequate resources to complement providers. Forecasting skills were referenced as a key point. “They need to be able to be assured that they can recruit more providers if needed,” a national support staff advisor noted.

Given the importance of developing effective relationships, communication skills were mentioned by most participants and underscored as critical to establishing trust between GPMs and others as the initiative was being implemented. Interviewees indicated that relationship building was further enhanced when GPMs possessed the ability to “work with” rather than command clinicians and staff; navigate politics; and were respectful of other people’s knowledge, skills, abilities, and status. “They have to work with the nursing staff and teach them,” a leader described, “so that people understand that we are going to a different place to achieve our primary objectives and goals.”

Issues and Challenges

Participants identified several technical, social, and structural challenges and barriers to successfully implementing the GPM role.

Technical challenges. Recurring themes across all phases of data analytics were GPMs’ capability to challenge data use and use large volumes of information from multiple data sources (entering and accessing data; “drilling down” from summaries; generating reports; and analyzing and interpreting resulting metrics). Interviewees reported that information assessment and analytic support were not consistent. One GPM had a data analyst pulling reports needed to support clinical units while other GPMs trained staff to pull data. Even with support, some GPMs had issues due to limited information technology (IT) skills or access privileges leading to inefficiencies and delays. “Whenever I need anything from a programmer, I have to go through, you know, the IT gods in the sky,” one GPM remarked. “That usually takes a few months or more.”

Social challenges. Instituting the GPM role was a cultural change, and interviewees reported needing to address resistance to CPM model efforts. Resistance to change “is particularly hard in the VA just because it has a unique culture,” one leader noted. “There is a comfort in the legacy way of doing things.” The GPM initiative was introduced during a time when other national level initiatives were being implemented throughout the VHA. Fulfilling requests for information for these initiatives became the responsibility of the GPM and their team, which diverted attention from the mandate to improve access. Furthermore, GPMs were often considered the “change communicators” to clinics putting them in the role of “bad messenger,” which degraded trust and made it difficult to partner with clinicians.

Efforts to work through change management and build relationships included general program awareness presentation to internal stakeholders; including key stakeholders in GPM committees; pre-emptive conversations with unit chiefs; creating awareness of the GPM activities and progress through formal and informal update meetings; and identifying successes regarding access.

Structural challenges. The GPM role did not have direct supervision over clinical and administrative leaders, making it challenging to enact change. GPMs reported that “they do not always have authority over the area that they are being asked to manage,” which made their work difficult, requiring strong negotiation skills and political savvy to affect change. However, as the clinic staff and providers saw how the GPM could support and positively impact their practice, these challenges began to subside.

Discussion

This study provides empirical evidence regarding the implementation of a new access management strategy for health care systems focused on improving timeliness of care. First, the GPM position was seen as critical at each facility, as a single point person, to help local system leaders respond effectively to both national mandates and local context. Second, requiring the GPMs to report to the medical center director or chief of staff was important for integrating access perspectives across service lines and to facilitate a strong GPM role in strategic planning. Third, the intentional flexibility of the access management initiative, beyond the nationally specified aspects of the GPM role, was key for allowing individual sites to adapt to unique local challenges, resources, and population demands. Fourth, the initiative provided GPMs with opportunities to learn important skills and support the acquisition, utilization, and communication of a tremendous range of data toward responsive action.

According to our respondents, the GPM role demands functioning across a broad set of responsibilities; understanding the big picture as well as the complex underlying variables; engaging facility leaders, clinical and administrative staff; and prioritizing competing national and local demands. Consistent with previous findings, effective GPMs must possess a complex set of skills (interpersonal, analytic, and leadership) and the ability to create a supporting team.¹³

In practice, improving access at individual sites of care (VA medical centers and community-based outpatient clinics) poses significant challenges, especially in the early stages, even with the assistance of a GPM. For example, some respondents reported being overwhelmed by the volume of available data and dashboards, and responding to current requests for data analysis and dissemination sometimes impeded long range planning. Multiple national access-related initiatives and local pressures also generated excessive and potentially conflicting demands. Thus, while the creation of a GPM position seemed to be essential for the pilot sites to improve local access and timeliness to care, success also requires ongoing national and facility-wide communication, education, and support. Ongoing data analysis training and support will be critical to ensuring the sustainability of the position. Last, recruiting GPMs with the needed complex skill set may prove to be challenging, and it will be important to provide resources to identify, attract, and retain well-qualified GPMs.

Limitations and Future Work

This study was based on a small initial sample of pilot sites of varying sizes and, as such, may not reflect the experience of all VHA GPMs. In addition, the use of snowball sampling, while facilitating identification of key stakeholders, may introduce bias in participant sampling. Nonetheless, the results from this study provide findings that informed the national VHA GPM initiative and can inform further studies of practice management roles outside of the VA.

Further study of the VHA GPM implementation and similar roles in other health care systems is needed. As the GPM position is fully implemented across the VHA, large scale evaluation is needed to gain a more representative picture and allow for comparison of the GPM role at various types of facilities (eg, size, rurality, complexity, ranking based on access performance metrics).

Conclusion

Improving access to care is a central goal for health care systems. The incorporation of the GPM role is an innovative approach to improve access management strategies. Early study of prototype sites provided VHA leadership with valuable insights used to influence further rollout of this initiative. Based on our findings, national and local support are important to ongoing success. National access mandates, training, and resources should focus on ensuring sufficient GPM authority, enabling GPMs to use data, and ensuring GPMs engage with frontline clinical and administrative staff to improve veteran access to care.