High utilizers of hospital services are medically complex, psychosocially vulnerable, and at risk for adverse health outcomes.[1, 2] They make up a fraction of the patient population but use a disproportionate amount of resources, with high rates of emergency department (ED) visits and hospital admissions.[1, 3, 4] Less than 1% of patients account for 21% of national healthcare spending, and hospital costs are the largest category of national healthcare expenditures.[2, 5] Many patients who disproportionately contribute to high healthcare costs also have high hospital admission rates.[6]

Interventions targeting high utilizers have typically focused on the outpatient setting.[7, 8, 9, 10] Interventions using individualized care plans in the ED reduced ED visits from 33% to 70%, but all have required an additional case management program or partnership with an outside nonprofit case management organization.[11, 12, 13] One study by a hospitalist group using individualized care plans reduced ED visits and admissions by 70%, 2 months after care‐plan implementation; however, all of their care plans were focused explicitly on restricting intravenous opiate use for patients with chronic pain.[14]

Given the current focus on cost‐conscious, high‐quality care in the American healthcare system, we designed a quality‐improvement (QI) intervention using individualized care plans to reduce unnecessary healthcare service utilization and hospital costs for the highest utilizers of ED and inpatient care. Our approach focuses on integrating care plans within our electronic medical record (EMR) and implementing them using the existing healthcare workforce. We analyzed pre‐ and postintervention data to determine its effect on service utilization and hospital costs across a regional health system.

METHODS

RESULTS

Table 1 shows the demographics and comorbidities for the 24 patients with care plans included in this analysis. The average age of patients is 38.5 years (range, 2565 years) and a nearly even split between males (11) and females (13). Chronic disease burden is high. Furthermore, 83% of patients have chronic pain and 96% have mental health problems or substance abuse.

Table 2 shows inpatient and ED utilization patterns before and after care‐plan implementation. Inpatient admissions decreased by 56% for the 6 months after care‐plan implementation (P<0.001) and by 50.5% for the 12 months after care‐plan implementation (P=0.003). This translates to a decrease in the average number of admissions per patient from 6.3 to 2.4, 6 months postcare plan, and from 10.9 to 4.8, 12 months postcare plan.

Thirty‐day readmissions also significantly decreased after care‐plan implementation. Among the 23 patients with data 6 months pre and postcare plan, there were 130 readmissions before and 44 readmissions after care‐plan implementation, a 66% reduction (P<0.001). Among the 12 patients with data 12 months pre and postcare plan, there were 106 readmissions before and 45 readmissions after care‐plan implementation, a 51.5% reduction (P=0.002). Inpatient LOS did not show a statistically significant change after care‐plan implementation.

ED visits were similar for the 6 months pre compared to 6 months postcare plan. ED visits at 12 months postcare plan increased from an average of 15.4 visits pre to 25.6 visits per patient postcare plan. This was driven by a single homeless patient with dialysis‐dependent end‐stage renal disease, who had 134 ED visits in the 12 months after careplan implementation. Analysis of the data with this outlier removed showed a reduction in ED visits from an average of 12.3 visits per patient to 10.1 visits per patient in the 12 months postcare plan; however, this was not statistically significant (P=0.66, data not shown).

Table 3 shows inpatient and ED VDCs before and after care‐plan implementation. The average VDCs per patient per admission decreased from $29,852.71 to $15,587.84, 6 months after care‐plan implementation, a 47.7% reduction (P=0.001). The average VDCs per patient per admission decreased from $44,881.66 to $24,958.42, 12 months after care‐plan implementation, a 35.8% reduction (P=0.052). ED costs did not show a statistically significant decrease. However, with the outlier removed as above, costs did decrease by 12.3%, 6 months after care‐plan implementation, approaching statistical significance (P=0.073, data not shown). Combined inpatient and ED variable direct costs decreased by an average of $15,117.30, 6 months after care‐plan implementation, a 45.3% reduction (P=0.002), and by an average of $18,604.50, 12 months after care‐plan implementation, a 25.5% reduction, although this did not reach statistical significance (P=0.129).

DISCUSSION

A multidisciplinary team at our academic medical center developed individualized care plans tailored to the specific medical and psychosocial complexities of high utilizers to reduce unnecessary service utilization and hospital costs. Postintervention analysis shows reduced inpatient admissions and 30‐day readmissions among this population by 50%. Furthermore, inpatient variable direct costs decreased by 47% for the 6 months following care‐plan implementation and by 35% for the 12 months following care‐plan implementation. This translates into a $347,696.97 cost savings for the 23 patients 6 months after care‐plan implementation, and a $223,253.89 cost savings for the 12 patients 12 months after care‐plan implementation. This reduction in utilization and cost was seen across all 3 hospitals in the Duke University Health System, including the only 2 hospitals in Durham, North Carolina. Unlike other urban areas, public transportation in our region is scarce, and the options for hospital shopping in central North Carolina are relatively limited. Although this study does not measure utilization in surrounding counties, we do not feel this occurred as we did not see a rise in requests for medical records nor attempts to contact Duke providers for questions on these patients as a result of our intervention. This, along with our regional health system outcome analysis, provides support that our intervention did not cause patients to seek care elsewhere and result in cost‐shifting to other facilities.

We hypothesize that our care plans may be responsible for decreased admissions and 30‐day readmissions through several mechanisms. By raising awareness of these patients' excessive hospital utilization patterns and making this information readily available through our EMR, providers in the ED may be more conscientious about their admission decisions. Problems that at face value seem acute, are often more chronic and can be better managed in the outpatient setting. Several care plans also explicitly recommend limiting unnecessary intravenous opiate use for chronic pain patients. Other patients who have frequent admissions actually have end‐stage disease, and care plans for these patients help facilitate referrals to hospice programs.

Care plans provide a consistent message of patient histories, utilization patterns, and management strategies, and also serve as a communication tool between hospitalists and ED providers. A systematic review of all ED‐based interventions for high utilizers revealed that most studies did show a reduction in ED visits, but all incorporated case management programs to do so.[15] We did not reduce ED visits, possibly because we lacked the resources and care coordination a community‐based case management program provides. However, care plans did serve as a platform with which hospitalists and ED providers can help coordinate care among multiple outpatient providers. This has potentially limited admissions by providing a referral destination or outpatient point of contact for ED providers. For example, as a result of our intervention, referral mechanisms to our comprehensive pain clinic and outpatient psychiatry clinic have both been strengthened and streamlined. The fact that care plans decreased admissions and readmissions, but not ED visits, suggests that our intervention may not have actually changed patient behavior, but instead changed provider practices in relation to disposition decisions in the ED.

Our QI intervention has several strengths. First, it is fully integrated within our existing healthcare workforce, without the need for an extra case management system. Second, it is seamlessly incorporated into our EMR and represents another potential use of an EMR that has not been previously touted. Third, the multidisciplinary nature of the CCPC ensures that all stakeholders involved in the care of high utilizers are represented. Fourth, the outcome analysis across all 3 hospitals in our health system provides a balancing metric against the notion that our intervention simply caused patients to seek care elsewhere in the region. Last, the QI design and lack of strict inclusion and exclusion criteria adds practicality and shows effectiveness, not just efficacy, of the intervention.

Because this was developed as a QI intervention without strict inclusion and exclusion criteria, generalizability is lacking. In the future, one could use the EMR to more systematically identify high‐utilizing, complex patients. One study showed the ability to use the EMR with a standardized framework to identify hot spotting (high utilizers) and contextual anomaly detection (ie, anomalous utilization cases where patient‐incurred levels of utilization are unexpected given their clinical characteristics).[16] The nonrandomized, retrospective pre/post‐intervention analysis without a control group diminishes the external validity of the results and does introduce the potential for bias.

One of the primary study limitations includes the small sample size of only 24 patients. Admittedly, these first 24 patients are the absolute highest utilizers of care at our hospital, possibly making their utilization patterns more amenable to our intervention. The 96% prevalence rate of mental health and substance abuse in our cohort is significantly higher than other published data among high utilizers.[4, 17, 18] We are continuing to develop care plans for additional high‐utilizing, complex patients, and expect to enroll more patients with end‐stage disease, and relatively fewer with substance abuse or psychiatric illness as time goes on. It is possible this new cohort of patients has proportionally less unnecessary utilization, thus limiting our intervention effect. One final limitation of our study is the lack of care quality and patient safety outcomes. In future studies, health outcomes, adverse events, and outpatient care utilization will be important balancing measures to include.

In conclusion, we showed that a QI intervention using individualized care plans reduces hospital admissions, 30‐day readmissions, and hospital costs across a regional health system for a group of complex, high‐utilizing patients. This intervention can, and should, be developed by a multidisciplinary team and fully integrated into the existing healthcare workforce and EMR to ensure appropriateness, effectiveness, and longevity. Going forward, it will be imperative to evaluate this intervention prospectively, at multiple sites, in coordination with outpatient providers, and including quality and safety outcomes to determine if this hospital‐based intervention impacts care coordination, utilization rates, cost, and health outcomes across the broader healthcare system.

High utilizers of hospital services are medically complex, psychosocially vulnerable, and at risk for adverse health outcomes.[1, 2] They make up a fraction of the patient population but use a disproportionate amount of resources, with high rates of emergency department (ED) visits and hospital admissions.[1, 3, 4] Less than 1% of patients account for 21% of national healthcare spending, and hospital costs are the largest category of national healthcare expenditures.[2, 5] Many patients who disproportionately contribute to high healthcare costs also have high hospital admission rates.[6]

Interventions targeting high utilizers have typically focused on the outpatient setting.[7, 8, 9, 10] Interventions using individualized care plans in the ED reduced ED visits from 33% to 70%, but all have required an additional case management program or partnership with an outside nonprofit case management organization.[11, 12, 13] One study by a hospitalist group using individualized care plans reduced ED visits and admissions by 70%, 2 months after care‐plan implementation; however, all of their care plans were focused explicitly on restricting intravenous opiate use for patients with chronic pain.[14]

Given the current focus on cost‐conscious, high‐quality care in the American healthcare system, we designed a quality‐improvement (QI) intervention using individualized care plans to reduce unnecessary healthcare service utilization and hospital costs for the highest utilizers of ED and inpatient care. Our approach focuses on integrating care plans within our electronic medical record (EMR) and implementing them using the existing healthcare workforce. We analyzed pre‐ and postintervention data to determine its effect on service utilization and hospital costs across a regional health system.

METHODS

RESULTS

Table 1 shows the demographics and comorbidities for the 24 patients with care plans included in this analysis. The average age of patients is 38.5 years (range, 2565 years) and a nearly even split between males (11) and females (13). Chronic disease burden is high. Furthermore, 83% of patients have chronic pain and 96% have mental health problems or substance abuse.

Table 2 shows inpatient and ED utilization patterns before and after care‐plan implementation. Inpatient admissions decreased by 56% for the 6 months after care‐plan implementation (P<0.001) and by 50.5% for the 12 months after care‐plan implementation (P=0.003). This translates to a decrease in the average number of admissions per patient from 6.3 to 2.4, 6 months postcare plan, and from 10.9 to 4.8, 12 months postcare plan.

Thirty‐day readmissions also significantly decreased after care‐plan implementation. Among the 23 patients with data 6 months pre and postcare plan, there were 130 readmissions before and 44 readmissions after care‐plan implementation, a 66% reduction (P<0.001). Among the 12 patients with data 12 months pre and postcare plan, there were 106 readmissions before and 45 readmissions after care‐plan implementation, a 51.5% reduction (P=0.002). Inpatient LOS did not show a statistically significant change after care‐plan implementation.

ED visits were similar for the 6 months pre compared to 6 months postcare plan. ED visits at 12 months postcare plan increased from an average of 15.4 visits pre to 25.6 visits per patient postcare plan. This was driven by a single homeless patient with dialysis‐dependent end‐stage renal disease, who had 134 ED visits in the 12 months after careplan implementation. Analysis of the data with this outlier removed showed a reduction in ED visits from an average of 12.3 visits per patient to 10.1 visits per patient in the 12 months postcare plan; however, this was not statistically significant (P=0.66, data not shown).

Table 3 shows inpatient and ED VDCs before and after care‐plan implementation. The average VDCs per patient per admission decreased from $29,852.71 to $15,587.84, 6 months after care‐plan implementation, a 47.7% reduction (P=0.001). The average VDCs per patient per admission decreased from $44,881.66 to $24,958.42, 12 months after care‐plan implementation, a 35.8% reduction (P=0.052). ED costs did not show a statistically significant decrease. However, with the outlier removed as above, costs did decrease by 12.3%, 6 months after care‐plan implementation, approaching statistical significance (P=0.073, data not shown). Combined inpatient and ED variable direct costs decreased by an average of $15,117.30, 6 months after care‐plan implementation, a 45.3% reduction (P=0.002), and by an average of $18,604.50, 12 months after care‐plan implementation, a 25.5% reduction, although this did not reach statistical significance (P=0.129).

DISCUSSION

A multidisciplinary team at our academic medical center developed individualized care plans tailored to the specific medical and psychosocial complexities of high utilizers to reduce unnecessary service utilization and hospital costs. Postintervention analysis shows reduced inpatient admissions and 30‐day readmissions among this population by 50%. Furthermore, inpatient variable direct costs decreased by 47% for the 6 months following care‐plan implementation and by 35% for the 12 months following care‐plan implementation. This translates into a $347,696.97 cost savings for the 23 patients 6 months after care‐plan implementation, and a $223,253.89 cost savings for the 12 patients 12 months after care‐plan implementation. This reduction in utilization and cost was seen across all 3 hospitals in the Duke University Health System, including the only 2 hospitals in Durham, North Carolina. Unlike other urban areas, public transportation in our region is scarce, and the options for hospital shopping in central North Carolina are relatively limited. Although this study does not measure utilization in surrounding counties, we do not feel this occurred as we did not see a rise in requests for medical records nor attempts to contact Duke providers for questions on these patients as a result of our intervention. This, along with our regional health system outcome analysis, provides support that our intervention did not cause patients to seek care elsewhere and result in cost‐shifting to other facilities.

We hypothesize that our care plans may be responsible for decreased admissions and 30‐day readmissions through several mechanisms. By raising awareness of these patients' excessive hospital utilization patterns and making this information readily available through our EMR, providers in the ED may be more conscientious about their admission decisions. Problems that at face value seem acute, are often more chronic and can be better managed in the outpatient setting. Several care plans also explicitly recommend limiting unnecessary intravenous opiate use for chronic pain patients. Other patients who have frequent admissions actually have end‐stage disease, and care plans for these patients help facilitate referrals to hospice programs.

Care plans provide a consistent message of patient histories, utilization patterns, and management strategies, and also serve as a communication tool between hospitalists and ED providers. A systematic review of all ED‐based interventions for high utilizers revealed that most studies did show a reduction in ED visits, but all incorporated case management programs to do so.[15] We did not reduce ED visits, possibly because we lacked the resources and care coordination a community‐based case management program provides. However, care plans did serve as a platform with which hospitalists and ED providers can help coordinate care among multiple outpatient providers. This has potentially limited admissions by providing a referral destination or outpatient point of contact for ED providers. For example, as a result of our intervention, referral mechanisms to our comprehensive pain clinic and outpatient psychiatry clinic have both been strengthened and streamlined. The fact that care plans decreased admissions and readmissions, but not ED visits, suggests that our intervention may not have actually changed patient behavior, but instead changed provider practices in relation to disposition decisions in the ED.

Our QI intervention has several strengths. First, it is fully integrated within our existing healthcare workforce, without the need for an extra case management system. Second, it is seamlessly incorporated into our EMR and represents another potential use of an EMR that has not been previously touted. Third, the multidisciplinary nature of the CCPC ensures that all stakeholders involved in the care of high utilizers are represented. Fourth, the outcome analysis across all 3 hospitals in our health system provides a balancing metric against the notion that our intervention simply caused patients to seek care elsewhere in the region. Last, the QI design and lack of strict inclusion and exclusion criteria adds practicality and shows effectiveness, not just efficacy, of the intervention.

Because this was developed as a QI intervention without strict inclusion and exclusion criteria, generalizability is lacking. In the future, one could use the EMR to more systematically identify high‐utilizing, complex patients. One study showed the ability to use the EMR with a standardized framework to identify hot spotting (high utilizers) and contextual anomaly detection (ie, anomalous utilization cases where patient‐incurred levels of utilization are unexpected given their clinical characteristics).[16] The nonrandomized, retrospective pre/post‐intervention analysis without a control group diminishes the external validity of the results and does introduce the potential for bias.

One of the primary study limitations includes the small sample size of only 24 patients. Admittedly, these first 24 patients are the absolute highest utilizers of care at our hospital, possibly making their utilization patterns more amenable to our intervention. The 96% prevalence rate of mental health and substance abuse in our cohort is significantly higher than other published data among high utilizers.[4, 17, 18] We are continuing to develop care plans for additional high‐utilizing, complex patients, and expect to enroll more patients with end‐stage disease, and relatively fewer with substance abuse or psychiatric illness as time goes on. It is possible this new cohort of patients has proportionally less unnecessary utilization, thus limiting our intervention effect. One final limitation of our study is the lack of care quality and patient safety outcomes. In future studies, health outcomes, adverse events, and outpatient care utilization will be important balancing measures to include.

In conclusion, we showed that a QI intervention using individualized care plans reduces hospital admissions, 30‐day readmissions, and hospital costs across a regional health system for a group of complex, high‐utilizing patients. This intervention can, and should, be developed by a multidisciplinary team and fully integrated into the existing healthcare workforce and EMR to ensure appropriateness, effectiveness, and longevity. Going forward, it will be imperative to evaluate this intervention prospectively, at multiple sites, in coordination with outpatient providers, and including quality and safety outcomes to determine if this hospital‐based intervention impacts care coordination, utilization rates, cost, and health outcomes across the broader healthcare system.

High utilizers of hospital services are medically complex, psychosocially vulnerable, and at risk for adverse health outcomes.[1, 2] They make up a fraction of the patient population but use a disproportionate amount of resources, with high rates of emergency department (ED) visits and hospital admissions.[1, 3, 4] Less than 1% of patients account for 21% of national healthcare spending, and hospital costs are the largest category of national healthcare expenditures.[2, 5] Many patients who disproportionately contribute to high healthcare costs also have high hospital admission rates.[6]

Interventions targeting high utilizers have typically focused on the outpatient setting.[7, 8, 9, 10] Interventions using individualized care plans in the ED reduced ED visits from 33% to 70%, but all have required an additional case management program or partnership with an outside nonprofit case management organization.[11, 12, 13] One study by a hospitalist group using individualized care plans reduced ED visits and admissions by 70%, 2 months after care‐plan implementation; however, all of their care plans were focused explicitly on restricting intravenous opiate use for patients with chronic pain.[14]

Given the current focus on cost‐conscious, high‐quality care in the American healthcare system, we designed a quality‐improvement (QI) intervention using individualized care plans to reduce unnecessary healthcare service utilization and hospital costs for the highest utilizers of ED and inpatient care. Our approach focuses on integrating care plans within our electronic medical record (EMR) and implementing them using the existing healthcare workforce. We analyzed pre‐ and postintervention data to determine its effect on service utilization and hospital costs across a regional health system.

METHODS

RESULTS

Table 1 shows the demographics and comorbidities for the 24 patients with care plans included in this analysis. The average age of patients is 38.5 years (range, 2565 years) and a nearly even split between males (11) and females (13). Chronic disease burden is high. Furthermore, 83% of patients have chronic pain and 96% have mental health problems or substance abuse.

Table 2 shows inpatient and ED utilization patterns before and after care‐plan implementation. Inpatient admissions decreased by 56% for the 6 months after care‐plan implementation (P<0.001) and by 50.5% for the 12 months after care‐plan implementation (P=0.003). This translates to a decrease in the average number of admissions per patient from 6.3 to 2.4, 6 months postcare plan, and from 10.9 to 4.8, 12 months postcare plan.

Thirty‐day readmissions also significantly decreased after care‐plan implementation. Among the 23 patients with data 6 months pre and postcare plan, there were 130 readmissions before and 44 readmissions after care‐plan implementation, a 66% reduction (P<0.001). Among the 12 patients with data 12 months pre and postcare plan, there were 106 readmissions before and 45 readmissions after care‐plan implementation, a 51.5% reduction (P=0.002). Inpatient LOS did not show a statistically significant change after care‐plan implementation.

ED visits were similar for the 6 months pre compared to 6 months postcare plan. ED visits at 12 months postcare plan increased from an average of 15.4 visits pre to 25.6 visits per patient postcare plan. This was driven by a single homeless patient with dialysis‐dependent end‐stage renal disease, who had 134 ED visits in the 12 months after careplan implementation. Analysis of the data with this outlier removed showed a reduction in ED visits from an average of 12.3 visits per patient to 10.1 visits per patient in the 12 months postcare plan; however, this was not statistically significant (P=0.66, data not shown).

Table 3 shows inpatient and ED VDCs before and after care‐plan implementation. The average VDCs per patient per admission decreased from $29,852.71 to $15,587.84, 6 months after care‐plan implementation, a 47.7% reduction (P=0.001). The average VDCs per patient per admission decreased from $44,881.66 to $24,958.42, 12 months after care‐plan implementation, a 35.8% reduction (P=0.052). ED costs did not show a statistically significant decrease. However, with the outlier removed as above, costs did decrease by 12.3%, 6 months after care‐plan implementation, approaching statistical significance (P=0.073, data not shown). Combined inpatient and ED variable direct costs decreased by an average of $15,117.30, 6 months after care‐plan implementation, a 45.3% reduction (P=0.002), and by an average of $18,604.50, 12 months after care‐plan implementation, a 25.5% reduction, although this did not reach statistical significance (P=0.129).

DISCUSSION

A multidisciplinary team at our academic medical center developed individualized care plans tailored to the specific medical and psychosocial complexities of high utilizers to reduce unnecessary service utilization and hospital costs. Postintervention analysis shows reduced inpatient admissions and 30‐day readmissions among this population by 50%. Furthermore, inpatient variable direct costs decreased by 47% for the 6 months following care‐plan implementation and by 35% for the 12 months following care‐plan implementation. This translates into a $347,696.97 cost savings for the 23 patients 6 months after care‐plan implementation, and a $223,253.89 cost savings for the 12 patients 12 months after care‐plan implementation. This reduction in utilization and cost was seen across all 3 hospitals in the Duke University Health System, including the only 2 hospitals in Durham, North Carolina. Unlike other urban areas, public transportation in our region is scarce, and the options for hospital shopping in central North Carolina are relatively limited. Although this study does not measure utilization in surrounding counties, we do not feel this occurred as we did not see a rise in requests for medical records nor attempts to contact Duke providers for questions on these patients as a result of our intervention. This, along with our regional health system outcome analysis, provides support that our intervention did not cause patients to seek care elsewhere and result in cost‐shifting to other facilities.

We hypothesize that our care plans may be responsible for decreased admissions and 30‐day readmissions through several mechanisms. By raising awareness of these patients' excessive hospital utilization patterns and making this information readily available through our EMR, providers in the ED may be more conscientious about their admission decisions. Problems that at face value seem acute, are often more chronic and can be better managed in the outpatient setting. Several care plans also explicitly recommend limiting unnecessary intravenous opiate use for chronic pain patients. Other patients who have frequent admissions actually have end‐stage disease, and care plans for these patients help facilitate referrals to hospice programs.

Care plans provide a consistent message of patient histories, utilization patterns, and management strategies, and also serve as a communication tool between hospitalists and ED providers. A systematic review of all ED‐based interventions for high utilizers revealed that most studies did show a reduction in ED visits, but all incorporated case management programs to do so.[15] We did not reduce ED visits, possibly because we lacked the resources and care coordination a community‐based case management program provides. However, care plans did serve as a platform with which hospitalists and ED providers can help coordinate care among multiple outpatient providers. This has potentially limited admissions by providing a referral destination or outpatient point of contact for ED providers. For example, as a result of our intervention, referral mechanisms to our comprehensive pain clinic and outpatient psychiatry clinic have both been strengthened and streamlined. The fact that care plans decreased admissions and readmissions, but not ED visits, suggests that our intervention may not have actually changed patient behavior, but instead changed provider practices in relation to disposition decisions in the ED.

Our QI intervention has several strengths. First, it is fully integrated within our existing healthcare workforce, without the need for an extra case management system. Second, it is seamlessly incorporated into our EMR and represents another potential use of an EMR that has not been previously touted. Third, the multidisciplinary nature of the CCPC ensures that all stakeholders involved in the care of high utilizers are represented. Fourth, the outcome analysis across all 3 hospitals in our health system provides a balancing metric against the notion that our intervention simply caused patients to seek care elsewhere in the region. Last, the QI design and lack of strict inclusion and exclusion criteria adds practicality and shows effectiveness, not just efficacy, of the intervention.

Because this was developed as a QI intervention without strict inclusion and exclusion criteria, generalizability is lacking. In the future, one could use the EMR to more systematically identify high‐utilizing, complex patients. One study showed the ability to use the EMR with a standardized framework to identify hot spotting (high utilizers) and contextual anomaly detection (ie, anomalous utilization cases where patient‐incurred levels of utilization are unexpected given their clinical characteristics).[16] The nonrandomized, retrospective pre/post‐intervention analysis without a control group diminishes the external validity of the results and does introduce the potential for bias.

One of the primary study limitations includes the small sample size of only 24 patients. Admittedly, these first 24 patients are the absolute highest utilizers of care at our hospital, possibly making their utilization patterns more amenable to our intervention. The 96% prevalence rate of mental health and substance abuse in our cohort is significantly higher than other published data among high utilizers.[4, 17, 18] We are continuing to develop care plans for additional high‐utilizing, complex patients, and expect to enroll more patients with end‐stage disease, and relatively fewer with substance abuse or psychiatric illness as time goes on. It is possible this new cohort of patients has proportionally less unnecessary utilization, thus limiting our intervention effect. One final limitation of our study is the lack of care quality and patient safety outcomes. In future studies, health outcomes, adverse events, and outpatient care utilization will be important balancing measures to include.

In conclusion, we showed that a QI intervention using individualized care plans reduces hospital admissions, 30‐day readmissions, and hospital costs across a regional health system for a group of complex, high‐utilizing patients. This intervention can, and should, be developed by a multidisciplinary team and fully integrated into the existing healthcare workforce and EMR to ensure appropriateness, effectiveness, and longevity. Going forward, it will be imperative to evaluate this intervention prospectively, at multiple sites, in coordination with outpatient providers, and including quality and safety outcomes to determine if this hospital‐based intervention impacts care coordination, utilization rates, cost, and health outcomes across the broader healthcare system.

Despite widespread efforts to predict 30‐day rehospitalizations among discharged general medical patients,[1, 2, 3] not many strategies have incorporated patient‐reported outcome (PRO) measures in predictive models.[4] This despite the many longitudinal studies of the ambulatory population that demonstrate the higher likelihood of hospitalizations among those who score poorly on General Self‐Rated Health (GSRH),[5, 6, 7] baseline or declining Health‐Related Quality of Life,[8, 9, 10, 11, 12] psychological symptoms,[13, 14] and physical symptoms assessments.[15] One of the few existing studies that included PRO measures in 30‐day readmission models showed the predictive value of the 12‐item short form (SF12) Physical Component Score.[16] Others showed that persistent symptoms were associated with readmissions in patients with heart disease.[17, 18]

The paucity of efforts to connect PRO measures to utilization may be due to the limited availability of these measures in routine clinical records and the incomplete knowledge about how various PRO measures may fluctuate during episodes of acute illnesses and their treatments during hospitalizations. Health perception measures reflect both enduring features like self‐concept as well as dynamic features like a person's immediate health status.[19] As such, GSRH reflects the presence of chronic illnesses but is also responsive to acute events.[20, 21] Similarly, Health‐Related Quality of Life measures are dynamic as they decline around episodes of acute illness but are stable over a longer time window in their tendency to recover.[22] We do not know how fluctuations in measures of symptom burden, perceived health, and quality of life around the hospital‐to‐home transition may differentially inform readmission risk. Using a longitudinal cohort study, we addressed 2 questions: (1) How do PRO measures change when measured serially during the hospital‐to‐home transition? (2) How does the relative timing of each PRO measure variably inform the risk of subsequent utilization events including hospital readmissions?

METHODS

We conducted a longitudinal cohort study using data originally collected for a trial (ClinicalTrials.gov Identifier NCT01391026) of an intervention that was shown to have no associations with variables evaluated in this study. Patients were recruited from the John H. Stroger Hospital of Cook County, an urban safety‐net hospital that serves 128 municipalities in northeastern Illinois including the City of Chicago. Patients were eligible if they (1) were admitted to the general medical wards, the medical intensive care unit, or the cardiac care unit between May 2011 and February 2012; (2) had a clinic appointment in the Hospital's general medicine clinic (GMC) in the prior 12 months to facilitate follow‐up; and (3) were able to communicate independently in English or Spanish. Randomly selected patients were approached during their hospitalization and consenting subjects completed an in‐person questionnaire on the day of discharge. Subjects were contacted by telephone around 30, 90, and 180 days thereafter to complete follow‐up questionnaires; we began calling patients around 2 weeks prior to the target day anticipating noncontact on the first attempts. All telephone interviews were conducted by research assistants who had no clinical training and who did not give care‐related advice to patients based on their survey response. A few patients whose follow‐up survey window straddled the date of a scheduled clinic appointment were invited to complete the questionnaire in the GMC's waiting area using computer kiosks enabled with audio computer‐assisted self‐interview technology described elsewhere.[23] The Charlson Comorbidity Index was calculated inclusive of diagnostic codes assigned over 3 months preceding the index hospitalization.[24]

The following instruments were administered at each interview. The physical symptom severity portion of the Memorial Symptom Assessment Scale (MSAS) solicited the severity rank (none/a little bit/somewhat/quite a bit/very much) of 17 physical symptoms in the last week; the score was calculated by averaging the severity rank of the 12 most common symptom in the sample.[25, 26] The Patient Reported Outcomes Measurement Information System (PROMIS) Global Health Short Form is an instrument assessing GSRH (1 item), Social Activities (1 item), Global Physical Health (4 items), and Global Mental Health (4 items including a single‐item quality‐of‐life measure). Fatigue and pain for Global Physical Health, and emotional health for Global Mental Health were assessed over the past 7 days. Each of the 2 Global Health scores was standardized to a national mean of 50 and standard deviation of 10.[27]

The rate of survey completion at each follow‐up was calculated. Characteristics of participants were tabulated. Characteristics of patients censored prior to study completion were compared with patients with complete data. Box plots for MSAS physical symptom severity, and Global Physical and Mental Health scores were constructed to illustrate the comparisons of the mean scores between each consecutive survey period using t tests assuming unequal variance. A similar box plot of GSRH illustrated the comparison of the median score between consecutive surveys using the rank sum test. Hospital‐based utilization events were defined as either an emergency department visit or hospitalization at 1 of the 2 hospitals of the Cook County Health & Hospitals System (CCHHS). After accounting for patient data censored due to death (date reported by family) or withdrawal from study, Kaplan‐Meier curves showing time to first hospital‐based utilization event during each interval between surveys were drawn separately for above‐ and below‐median MSAS, Global Physical and Mental Health scores, and for poor or fair versus good, very good, or excellent GSRH assessment. The null hypothesis that the survivor functions were equal between the better and worse median quantiles or GSRH categories was tested using the log‐rank test at 14 and 30 days from survey completion. Hazard ratios for time to first utilization event within 14 days of each survey were calculated for the MSAS score, Global Physical and Mental Health as continuous variables, and GSRH response categories relative to poor using bivariate and multivariate Cox proportional hazard equations. Multivariate models incorporated the following 5 covariates: at least 1 utilization event during the study period prior to the survey, Charlson score, age, gender, and race/ethnicity category. Likelihood ratio statistics were calculated to test the hypothesis that the model including the PRO measure and covariates predicted the outcome equally well compared to the nested model with only covariates. We used the traditional threshold of .05 when reporting significance. All analyses were performed in Stata 13 (StataCorp, College Station, TX). The methods for patient consent, data collection, analyses, and reporting were reviewed and approved by the CCHHS institutional review board.

RESULTS

A total of 196 patients completed the initial survey. The completion rates were 98%, 90%, and 88% for the 30‐, 90‐, and 180‐day follow‐up surveys, respectively. As shown in Table 1, participants average age was 52 years, and about half were women. The majority was non‐Hispanic black, and 21% preferred to complete the survey in Spanish. Diabetes, congestive heart failure, cancer, and chronic pulmonary disease were each prevalent in at least one‐fifth of our patient cohort. Demographic characteristics were similar between the 160 patients who completed all 3 follow‐up surveys and the 36 who missed at least 1 follow‐up survey. Among the latter group, 1 withdrew at 30 days, 1 withdrew and 4 had died at 90 days, and 1 withdrew and 9 had died at 180 days.

Figure 1 shows a timeline of the follow‐up surveys and utilization events in the form of overlapping histograms. The majority of 30‐day follow‐up questionnaires were completed earlier than targeted, at a median of 17 (interquartile range [IQR] 16, 20) days after discharge. Similarly, questionnaires targeted for 90 and 180 days were completed at medians of 78 (IQR 7684) and 167 (IQR 166169) days from discharge. Fifty‐four (28%) patients experienced a first utilization event in the first 30 days following discharge. During the 60‐, 90‐, and 30‐day intervals after the first, second, and third follow‐up surveys, respectively, 63 (33%), 54 (31%), and 16 (9%) patients experienced a first utilization event.

Figure 1

A significant improvement in MSAS physical symptom severity was detected between the hospitalization and the 30‐day follow‐up (Figure 2A). Although the mean Global Physical Health score was below the national mean of 50 at every survey period, a similar improvement in the measure was noted between the hospitalization and the 30‐day follow‐up (Figure 2B). The mean Global Mental Health score was also below the national mean but remained stable throughout the study (Figure 2C). The median GSRH was stable at 2 (IQR 23) at every survey wave (Figure 2D). Of note, compared to patients who completed all 3 follow‐up surveys, patients who missed at least 1 follow‐up reported higher MSAS score (1.5 vs 1.8, P=0.03), lower Global Physical Health (36.1 vs 33.5, P=0.09), and lower Global Mental Health (44.7 vs 41.0, P=0.03) during their hospitalization. In addition, patients with complete data experienced an average of 1.2 utilization events during the study, whereas those with missing data experienced an average of 2.1 utilization events (P=0.03).

Figure 2

The MSAS physical symptom severity and Global Physical Health scores from the index hospitalizations did not identify patients with a first utilization event within 30 days. However, patients with poor Global Mental Health and GSRH in the hospital were more likely to experience a utilization event within 14 days of discharge (Figure 3). During the postdischarge period, patients scoring poorly on each of the PRO measures trended toward a greater risk of an early utilization event, but the association between utilization and MSAS was most consistently significant (Figure 3A). In general, the associations with MSAS, Global Physical Health, and GSRH were stronger with the risk of utilization events within 14 days than within 30 days (Figure 3A,B,D). The Global Mental Health score was not associated with a subsequent utilization when measured during the 180‐day postdischarge period.

Figure 3

As shown in Table 2, Cox proportional hazard models incorporating covariates preserved most of the significant associations seen in the unadjusted analyses. Global Mental Health and good relative to poor GSRH obtained during the hospitalization remained significant. MSAS obtained at each postdischarge follow‐up trended positively with utilization and was statistically significant at 90 and 180 days. Global Physical Health obtained at each postdischarge follow‐up similarly trended negatively with utilization and was significant at 180 days. Each multivariate model incorporating a PRO measure with a significant coefficient contributed to better fit of the predictive model compared to the nested model without the PRO measure.

DISCUSSION

In this longitudinal cohort study, patients, on average, reported relatively severe symptoms, low PROMIS Global Physical and Mental Health scores, and poor GSRH during the inpatient stay in an urban safety‐net hospital. Symptom severity and Global Physical Health improved, on average, by 30 days before stabilizing, but their poor levels in the hospital did not predict 30‐day hospital‐based utilization events. On the other hand, Global Mental Health and GSRH were stable through hospitalizations, and patients scoring poorly on these measures were at greater risk of utilization events within 14 days of discharge. PRO measures obtained during the 180‐day postdischarge period trended toward distinguishing populations with greater baseline risk of proximate utilization events. However, MSAS physical symptom severity and Global Physical Health were more consistently predictive of these events at statistically significant levels compared to Global Mental Health and GSRH in our relatively small sample of patients. Each of these measures selectively improved the fit‐of‐risk prediction models for hospital‐based utilization.

Some of the heterogeneity in readmission risk is explained by differences in PRO measures. Although the MSAS score and Global Physical Health assessment were reliable predictors of utilization when measured in ambulatory settings, they were less discriminating during acute hospitalizations when everyone, on average, reported severe symptoms and poor function. Our results were consistent with other studies that demonstrated the fairly rapid recovery in symptoms that follow hospitalizations,[28, 29] and these measures may become informative of utilization risk as early as 2 weeks postdischarge. GSRH and Global Mental Health (a measure of health‐related quality of life) only predicted utilizations immediately at hospital discharge. As multidimensional measures that reflect physical, social, and emotional capacity, these measures may indicate vulnerabilities in patients least able to handle the stresses of the early postdischarge period.

There is growing momentum around collecting PRO measures in routine clinical care as quality indicators that capture patient‐centered concerns.[30] Our study explored a novel application of these measures whose routine collection will likely proliferate, not solely for the purpose of helping healthcare systems identify patients at risk of unplanned resource utilization. Although multidimensional PRO measures seldom reflect conditions directly modifiable by simple interventions, we believe that the association between physical symptom burden and utilization in our data reveals a possible target for practice improvement. Hospitalists have contributed enormously to shorter lengths of stay that risk sicker and quicker discharges.[31] To mitigate its potential side effects on symptom management, a discharge plan that acknowledges physical symptoms that sometimes persist or recur beyond the hospitalization may be appropriate. This may be accomplished by ensuring that acute symptoms are resolving, giving clear instructions for symptom management at home, as now the standard of care for conditions like asthma,[32] and explicitly communicating the presence of residual symptoms to providers entrusted with continuity care. As an effective feedback measure that can drive continuous quality improvement, we believe that a technology‐based surveillance strategy that spans both the inpatient and outpatient domains is necessary.[23]

There are some notable similarities and differences between the results of our study and a recent hospital‐based study of PRO measures that used data from the Multi‐Center Hospitalist Project.[16] The Physical Component Score of the SF12 is similar to the PROMIS Global Physical Health score in that both incorporate measures of physical function, perceived health, pain, and energy level. Curiously, the SF12 Physical Component Score, but not the PROMIS Global Physical Health score, was associated with 30‐day rehospitalizations. An important difference between the measures is where the SF12 asks about limitations during the past 4 weeks the PROMIS instrument inquires about physical function in general and levels of fatigue and pain in the past 7 days. Considering most hospitalizations last <7 days, the PROMIS instrument may better reflect the declines associated with the acute illness related to the hospitalization than the SF12 score. Additionally, the discrepancy between the association between hospital‐based GSRH and utilization in our study and the absence, thereof, in Hasan et al. is noteworthy. The difference here may be explained by their use of a 0‐ to 100‐point response scale in contrast to our study's verbally labeled 5‐point scale in the PROMIS instrument. The range of rating scales for survey questions is traditionally governed by the tension between the difficulty with mapping respondents judgment on an excessively large scale on one hand, and the failure of insufficient response options to discriminate between respondents with different underlying judgment on the other.[33] We suspect the former to be a drawback of the unlabeled 100‐point response scale, and conjecture that an association might be found in the Multi‐Center Hospitalist Study data if the responses were grouped into summative categories.

We recognize several limitations in our study. The first is the generalizability of our patient population to others, not insignificantly because of the high proportion of the uninsured (around 70% during the study period) and racial/ethnic minorities among them. Although utilization patterns are clearly affected by socioeconomic status,[34] there may also be differences in the way validated PRO measures are calibrated between patients of public and private healthcare systems.[35] Another limitation is our inability to count utilization events at institutions outside of the CCHHS during our study. However, because the study was conducted prior to Cook County's Medicaid expansion demonstration program as part of the Affordable Care Act,[36] many patients established in our system faced barriers to receiving nonemergency care outside of the CCHHS supporting our assumption that few of our patients were discharged from other hospitals. Causality cannot be established in observational studies. Consequently, high prior‐symptom burden may be associated with utilizations through unmeasured variables. Measures of symptom burden are vulnerable to overendorsement and amplification.[37, 38] Inferences based on statistical significance are affected by sample size, and our conclusions may change if conducted with a larger number of participants. Our response rates were excellent through the survey waves, but we did not achieve perfect follow‐up. Worse levels of PRO responses and higher levels of utilization among censored patients biased our results toward the null. Finally, although we did not find any predominant comorbidities associated with hospital‐based utilizations in our sample, our analyses may be vulnerable to inadequate control for illness severity, which may also have biased our results.

PRO measures are likely to be useful in clinical medicine.[39] But to fully apply the powers of PROs in informing clinically and operationally relevant outcomes, we must actively develop a system for obtaining these measures in routine clinical care. The availability of patient downtime makes hospitalizations conducive to gathering patient‐generated data, and may further enhance patient‐provider communication if survey output was readily available in electronic medical records. Exploring innovative strategies for collecting PROs in the hospital and beyond remains our future work.

Despite widespread efforts to predict 30‐day rehospitalizations among discharged general medical patients,[1, 2, 3] not many strategies have incorporated patient‐reported outcome (PRO) measures in predictive models.[4] This despite the many longitudinal studies of the ambulatory population that demonstrate the higher likelihood of hospitalizations among those who score poorly on General Self‐Rated Health (GSRH),[5, 6, 7] baseline or declining Health‐Related Quality of Life,[8, 9, 10, 11, 12] psychological symptoms,[13, 14] and physical symptoms assessments.[15] One of the few existing studies that included PRO measures in 30‐day readmission models showed the predictive value of the 12‐item short form (SF12) Physical Component Score.[16] Others showed that persistent symptoms were associated with readmissions in patients with heart disease.[17, 18]

The paucity of efforts to connect PRO measures to utilization may be due to the limited availability of these measures in routine clinical records and the incomplete knowledge about how various PRO measures may fluctuate during episodes of acute illnesses and their treatments during hospitalizations. Health perception measures reflect both enduring features like self‐concept as well as dynamic features like a person's immediate health status.[19] As such, GSRH reflects the presence of chronic illnesses but is also responsive to acute events.[20, 21] Similarly, Health‐Related Quality of Life measures are dynamic as they decline around episodes of acute illness but are stable over a longer time window in their tendency to recover.[22] We do not know how fluctuations in measures of symptom burden, perceived health, and quality of life around the hospital‐to‐home transition may differentially inform readmission risk. Using a longitudinal cohort study, we addressed 2 questions: (1) How do PRO measures change when measured serially during the hospital‐to‐home transition? (2) How does the relative timing of each PRO measure variably inform the risk of subsequent utilization events including hospital readmissions?

METHODS

We conducted a longitudinal cohort study using data originally collected for a trial (ClinicalTrials.gov Identifier NCT01391026) of an intervention that was shown to have no associations with variables evaluated in this study. Patients were recruited from the John H. Stroger Hospital of Cook County, an urban safety‐net hospital that serves 128 municipalities in northeastern Illinois including the City of Chicago. Patients were eligible if they (1) were admitted to the general medical wards, the medical intensive care unit, or the cardiac care unit between May 2011 and February 2012; (2) had a clinic appointment in the Hospital's general medicine clinic (GMC) in the prior 12 months to facilitate follow‐up; and (3) were able to communicate independently in English or Spanish. Randomly selected patients were approached during their hospitalization and consenting subjects completed an in‐person questionnaire on the day of discharge. Subjects were contacted by telephone around 30, 90, and 180 days thereafter to complete follow‐up questionnaires; we began calling patients around 2 weeks prior to the target day anticipating noncontact on the first attempts. All telephone interviews were conducted by research assistants who had no clinical training and who did not give care‐related advice to patients based on their survey response. A few patients whose follow‐up survey window straddled the date of a scheduled clinic appointment were invited to complete the questionnaire in the GMC's waiting area using computer kiosks enabled with audio computer‐assisted self‐interview technology described elsewhere.[23] The Charlson Comorbidity Index was calculated inclusive of diagnostic codes assigned over 3 months preceding the index hospitalization.[24]

The following instruments were administered at each interview. The physical symptom severity portion of the Memorial Symptom Assessment Scale (MSAS) solicited the severity rank (none/a little bit/somewhat/quite a bit/very much) of 17 physical symptoms in the last week; the score was calculated by averaging the severity rank of the 12 most common symptom in the sample.[25, 26] The Patient Reported Outcomes Measurement Information System (PROMIS) Global Health Short Form is an instrument assessing GSRH (1 item), Social Activities (1 item), Global Physical Health (4 items), and Global Mental Health (4 items including a single‐item quality‐of‐life measure). Fatigue and pain for Global Physical Health, and emotional health for Global Mental Health were assessed over the past 7 days. Each of the 2 Global Health scores was standardized to a national mean of 50 and standard deviation of 10.[27]

The rate of survey completion at each follow‐up was calculated. Characteristics of participants were tabulated. Characteristics of patients censored prior to study completion were compared with patients with complete data. Box plots for MSAS physical symptom severity, and Global Physical and Mental Health scores were constructed to illustrate the comparisons of the mean scores between each consecutive survey period using t tests assuming unequal variance. A similar box plot of GSRH illustrated the comparison of the median score between consecutive surveys using the rank sum test. Hospital‐based utilization events were defined as either an emergency department visit or hospitalization at 1 of the 2 hospitals of the Cook County Health & Hospitals System (CCHHS). After accounting for patient data censored due to death (date reported by family) or withdrawal from study, Kaplan‐Meier curves showing time to first hospital‐based utilization event during each interval between surveys were drawn separately for above‐ and below‐median MSAS, Global Physical and Mental Health scores, and for poor or fair versus good, very good, or excellent GSRH assessment. The null hypothesis that the survivor functions were equal between the better and worse median quantiles or GSRH categories was tested using the log‐rank test at 14 and 30 days from survey completion. Hazard ratios for time to first utilization event within 14 days of each survey were calculated for the MSAS score, Global Physical and Mental Health as continuous variables, and GSRH response categories relative to poor using bivariate and multivariate Cox proportional hazard equations. Multivariate models incorporated the following 5 covariates: at least 1 utilization event during the study period prior to the survey, Charlson score, age, gender, and race/ethnicity category. Likelihood ratio statistics were calculated to test the hypothesis that the model including the PRO measure and covariates predicted the outcome equally well compared to the nested model with only covariates. We used the traditional threshold of .05 when reporting significance. All analyses were performed in Stata 13 (StataCorp, College Station, TX). The methods for patient consent, data collection, analyses, and reporting were reviewed and approved by the CCHHS institutional review board.

RESULTS

A total of 196 patients completed the initial survey. The completion rates were 98%, 90%, and 88% for the 30‐, 90‐, and 180‐day follow‐up surveys, respectively. As shown in Table 1, participants average age was 52 years, and about half were women. The majority was non‐Hispanic black, and 21% preferred to complete the survey in Spanish. Diabetes, congestive heart failure, cancer, and chronic pulmonary disease were each prevalent in at least one‐fifth of our patient cohort. Demographic characteristics were similar between the 160 patients who completed all 3 follow‐up surveys and the 36 who missed at least 1 follow‐up survey. Among the latter group, 1 withdrew at 30 days, 1 withdrew and 4 had died at 90 days, and 1 withdrew and 9 had died at 180 days.

Figure 1 shows a timeline of the follow‐up surveys and utilization events in the form of overlapping histograms. The majority of 30‐day follow‐up questionnaires were completed earlier than targeted, at a median of 17 (interquartile range [IQR] 16, 20) days after discharge. Similarly, questionnaires targeted for 90 and 180 days were completed at medians of 78 (IQR 7684) and 167 (IQR 166169) days from discharge. Fifty‐four (28%) patients experienced a first utilization event in the first 30 days following discharge. During the 60‐, 90‐, and 30‐day intervals after the first, second, and third follow‐up surveys, respectively, 63 (33%), 54 (31%), and 16 (9%) patients experienced a first utilization event.

Figure 1

A significant improvement in MSAS physical symptom severity was detected between the hospitalization and the 30‐day follow‐up (Figure 2A). Although the mean Global Physical Health score was below the national mean of 50 at every survey period, a similar improvement in the measure was noted between the hospitalization and the 30‐day follow‐up (Figure 2B). The mean Global Mental Health score was also below the national mean but remained stable throughout the study (Figure 2C). The median GSRH was stable at 2 (IQR 23) at every survey wave (Figure 2D). Of note, compared to patients who completed all 3 follow‐up surveys, patients who missed at least 1 follow‐up reported higher MSAS score (1.5 vs 1.8, P=0.03), lower Global Physical Health (36.1 vs 33.5, P=0.09), and lower Global Mental Health (44.7 vs 41.0, P=0.03) during their hospitalization. In addition, patients with complete data experienced an average of 1.2 utilization events during the study, whereas those with missing data experienced an average of 2.1 utilization events (P=0.03).

Figure 2

The MSAS physical symptom severity and Global Physical Health scores from the index hospitalizations did not identify patients with a first utilization event within 30 days. However, patients with poor Global Mental Health and GSRH in the hospital were more likely to experience a utilization event within 14 days of discharge (Figure 3). During the postdischarge period, patients scoring poorly on each of the PRO measures trended toward a greater risk of an early utilization event, but the association between utilization and MSAS was most consistently significant (Figure 3A). In general, the associations with MSAS, Global Physical Health, and GSRH were stronger with the risk of utilization events within 14 days than within 30 days (Figure 3A,B,D). The Global Mental Health score was not associated with a subsequent utilization when measured during the 180‐day postdischarge period.

Figure 3

As shown in Table 2, Cox proportional hazard models incorporating covariates preserved most of the significant associations seen in the unadjusted analyses. Global Mental Health and good relative to poor GSRH obtained during the hospitalization remained significant. MSAS obtained at each postdischarge follow‐up trended positively with utilization and was statistically significant at 90 and 180 days. Global Physical Health obtained at each postdischarge follow‐up similarly trended negatively with utilization and was significant at 180 days. Each multivariate model incorporating a PRO measure with a significant coefficient contributed to better fit of the predictive model compared to the nested model without the PRO measure.

DISCUSSION

In this longitudinal cohort study, patients, on average, reported relatively severe symptoms, low PROMIS Global Physical and Mental Health scores, and poor GSRH during the inpatient stay in an urban safety‐net hospital. Symptom severity and Global Physical Health improved, on average, by 30 days before stabilizing, but their poor levels in the hospital did not predict 30‐day hospital‐based utilization events. On the other hand, Global Mental Health and GSRH were stable through hospitalizations, and patients scoring poorly on these measures were at greater risk of utilization events within 14 days of discharge. PRO measures obtained during the 180‐day postdischarge period trended toward distinguishing populations with greater baseline risk of proximate utilization events. However, MSAS physical symptom severity and Global Physical Health were more consistently predictive of these events at statistically significant levels compared to Global Mental Health and GSRH in our relatively small sample of patients. Each of these measures selectively improved the fit‐of‐risk prediction models for hospital‐based utilization.

Some of the heterogeneity in readmission risk is explained by differences in PRO measures. Although the MSAS score and Global Physical Health assessment were reliable predictors of utilization when measured in ambulatory settings, they were less discriminating during acute hospitalizations when everyone, on average, reported severe symptoms and poor function. Our results were consistent with other studies that demonstrated the fairly rapid recovery in symptoms that follow hospitalizations,[28, 29] and these measures may become informative of utilization risk as early as 2 weeks postdischarge. GSRH and Global Mental Health (a measure of health‐related quality of life) only predicted utilizations immediately at hospital discharge. As multidimensional measures that reflect physical, social, and emotional capacity, these measures may indicate vulnerabilities in patients least able to handle the stresses of the early postdischarge period.

There is growing momentum around collecting PRO measures in routine clinical care as quality indicators that capture patient‐centered concerns.[30] Our study explored a novel application of these measures whose routine collection will likely proliferate, not solely for the purpose of helping healthcare systems identify patients at risk of unplanned resource utilization. Although multidimensional PRO measures seldom reflect conditions directly modifiable by simple interventions, we believe that the association between physical symptom burden and utilization in our data reveals a possible target for practice improvement. Hospitalists have contributed enormously to shorter lengths of stay that risk sicker and quicker discharges.[31] To mitigate its potential side effects on symptom management, a discharge plan that acknowledges physical symptoms that sometimes persist or recur beyond the hospitalization may be appropriate. This may be accomplished by ensuring that acute symptoms are resolving, giving clear instructions for symptom management at home, as now the standard of care for conditions like asthma,[32] and explicitly communicating the presence of residual symptoms to providers entrusted with continuity care. As an effective feedback measure that can drive continuous quality improvement, we believe that a technology‐based surveillance strategy that spans both the inpatient and outpatient domains is necessary.[23]

There are some notable similarities and differences between the results of our study and a recent hospital‐based study of PRO measures that used data from the Multi‐Center Hospitalist Project.[16] The Physical Component Score of the SF12 is similar to the PROMIS Global Physical Health score in that both incorporate measures of physical function, perceived health, pain, and energy level. Curiously, the SF12 Physical Component Score, but not the PROMIS Global Physical Health score, was associated with 30‐day rehospitalizations. An important difference between the measures is where the SF12 asks about limitations during the past 4 weeks the PROMIS instrument inquires about physical function in general and levels of fatigue and pain in the past 7 days. Considering most hospitalizations last <7 days, the PROMIS instrument may better reflect the declines associated with the acute illness related to the hospitalization than the SF12 score. Additionally, the discrepancy between the association between hospital‐based GSRH and utilization in our study and the absence, thereof, in Hasan et al. is noteworthy. The difference here may be explained by their use of a 0‐ to 100‐point response scale in contrast to our study's verbally labeled 5‐point scale in the PROMIS instrument. The range of rating scales for survey questions is traditionally governed by the tension between the difficulty with mapping respondents judgment on an excessively large scale on one hand, and the failure of insufficient response options to discriminate between respondents with different underlying judgment on the other.[33] We suspect the former to be a drawback of the unlabeled 100‐point response scale, and conjecture that an association might be found in the Multi‐Center Hospitalist Study data if the responses were grouped into summative categories.

We recognize several limitations in our study. The first is the generalizability of our patient population to others, not insignificantly because of the high proportion of the uninsured (around 70% during the study period) and racial/ethnic minorities among them. Although utilization patterns are clearly affected by socioeconomic status,[34] there may also be differences in the way validated PRO measures are calibrated between patients of public and private healthcare systems.[35] Another limitation is our inability to count utilization events at institutions outside of the CCHHS during our study. However, because the study was conducted prior to Cook County's Medicaid expansion demonstration program as part of the Affordable Care Act,[36] many patients established in our system faced barriers to receiving nonemergency care outside of the CCHHS supporting our assumption that few of our patients were discharged from other hospitals. Causality cannot be established in observational studies. Consequently, high prior‐symptom burden may be associated with utilizations through unmeasured variables. Measures of symptom burden are vulnerable to overendorsement and amplification.[37, 38] Inferences based on statistical significance are affected by sample size, and our conclusions may change if conducted with a larger number of participants. Our response rates were excellent through the survey waves, but we did not achieve perfect follow‐up. Worse levels of PRO responses and higher levels of utilization among censored patients biased our results toward the null. Finally, although we did not find any predominant comorbidities associated with hospital‐based utilizations in our sample, our analyses may be vulnerable to inadequate control for illness severity, which may also have biased our results.

PRO measures are likely to be useful in clinical medicine.[39] But to fully apply the powers of PROs in informing clinically and operationally relevant outcomes, we must actively develop a system for obtaining these measures in routine clinical care. The availability of patient downtime makes hospitalizations conducive to gathering patient‐generated data, and may further enhance patient‐provider communication if survey output was readily available in electronic medical records. Exploring innovative strategies for collecting PROs in the hospital and beyond remains our future work.

Despite widespread efforts to predict 30‐day rehospitalizations among discharged general medical patients,[1, 2, 3] not many strategies have incorporated patient‐reported outcome (PRO) measures in predictive models.[4] This despite the many longitudinal studies of the ambulatory population that demonstrate the higher likelihood of hospitalizations among those who score poorly on General Self‐Rated Health (GSRH),[5, 6, 7] baseline or declining Health‐Related Quality of Life,[8, 9, 10, 11, 12] psychological symptoms,[13, 14] and physical symptoms assessments.[15] One of the few existing studies that included PRO measures in 30‐day readmission models showed the predictive value of the 12‐item short form (SF12) Physical Component Score.[16] Others showed that persistent symptoms were associated with readmissions in patients with heart disease.[17, 18]

The paucity of efforts to connect PRO measures to utilization may be due to the limited availability of these measures in routine clinical records and the incomplete knowledge about how various PRO measures may fluctuate during episodes of acute illnesses and their treatments during hospitalizations. Health perception measures reflect both enduring features like self‐concept as well as dynamic features like a person's immediate health status.[19] As such, GSRH reflects the presence of chronic illnesses but is also responsive to acute events.[20, 21] Similarly, Health‐Related Quality of Life measures are dynamic as they decline around episodes of acute illness but are stable over a longer time window in their tendency to recover.[22] We do not know how fluctuations in measures of symptom burden, perceived health, and quality of life around the hospital‐to‐home transition may differentially inform readmission risk. Using a longitudinal cohort study, we addressed 2 questions: (1) How do PRO measures change when measured serially during the hospital‐to‐home transition? (2) How does the relative timing of each PRO measure variably inform the risk of subsequent utilization events including hospital readmissions?

METHODS

We conducted a longitudinal cohort study using data originally collected for a trial (ClinicalTrials.gov Identifier NCT01391026) of an intervention that was shown to have no associations with variables evaluated in this study. Patients were recruited from the John H. Stroger Hospital of Cook County, an urban safety‐net hospital that serves 128 municipalities in northeastern Illinois including the City of Chicago. Patients were eligible if they (1) were admitted to the general medical wards, the medical intensive care unit, or the cardiac care unit between May 2011 and February 2012; (2) had a clinic appointment in the Hospital's general medicine clinic (GMC) in the prior 12 months to facilitate follow‐up; and (3) were able to communicate independently in English or Spanish. Randomly selected patients were approached during their hospitalization and consenting subjects completed an in‐person questionnaire on the day of discharge. Subjects were contacted by telephone around 30, 90, and 180 days thereafter to complete follow‐up questionnaires; we began calling patients around 2 weeks prior to the target day anticipating noncontact on the first attempts. All telephone interviews were conducted by research assistants who had no clinical training and who did not give care‐related advice to patients based on their survey response. A few patients whose follow‐up survey window straddled the date of a scheduled clinic appointment were invited to complete the questionnaire in the GMC's waiting area using computer kiosks enabled with audio computer‐assisted self‐interview technology described elsewhere.[23] The Charlson Comorbidity Index was calculated inclusive of diagnostic codes assigned over 3 months preceding the index hospitalization.[24]

The following instruments were administered at each interview. The physical symptom severity portion of the Memorial Symptom Assessment Scale (MSAS) solicited the severity rank (none/a little bit/somewhat/quite a bit/very much) of 17 physical symptoms in the last week; the score was calculated by averaging the severity rank of the 12 most common symptom in the sample.[25, 26] The Patient Reported Outcomes Measurement Information System (PROMIS) Global Health Short Form is an instrument assessing GSRH (1 item), Social Activities (1 item), Global Physical Health (4 items), and Global Mental Health (4 items including a single‐item quality‐of‐life measure). Fatigue and pain for Global Physical Health, and emotional health for Global Mental Health were assessed over the past 7 days. Each of the 2 Global Health scores was standardized to a national mean of 50 and standard deviation of 10.[27]

The rate of survey completion at each follow‐up was calculated. Characteristics of participants were tabulated. Characteristics of patients censored prior to study completion were compared with patients with complete data. Box plots for MSAS physical symptom severity, and Global Physical and Mental Health scores were constructed to illustrate the comparisons of the mean scores between each consecutive survey period using t tests assuming unequal variance. A similar box plot of GSRH illustrated the comparison of the median score between consecutive surveys using the rank sum test. Hospital‐based utilization events were defined as either an emergency department visit or hospitalization at 1 of the 2 hospitals of the Cook County Health & Hospitals System (CCHHS). After accounting for patient data censored due to death (date reported by family) or withdrawal from study, Kaplan‐Meier curves showing time to first hospital‐based utilization event during each interval between surveys were drawn separately for above‐ and below‐median MSAS, Global Physical and Mental Health scores, and for poor or fair versus good, very good, or excellent GSRH assessment. The null hypothesis that the survivor functions were equal between the better and worse median quantiles or GSRH categories was tested using the log‐rank test at 14 and 30 days from survey completion. Hazard ratios for time to first utilization event within 14 days of each survey were calculated for the MSAS score, Global Physical and Mental Health as continuous variables, and GSRH response categories relative to poor using bivariate and multivariate Cox proportional hazard equations. Multivariate models incorporated the following 5 covariates: at least 1 utilization event during the study period prior to the survey, Charlson score, age, gender, and race/ethnicity category. Likelihood ratio statistics were calculated to test the hypothesis that the model including the PRO measure and covariates predicted the outcome equally well compared to the nested model with only covariates. We used the traditional threshold of .05 when reporting significance. All analyses were performed in Stata 13 (StataCorp, College Station, TX). The methods for patient consent, data collection, analyses, and reporting were reviewed and approved by the CCHHS institutional review board.

RESULTS

A total of 196 patients completed the initial survey. The completion rates were 98%, 90%, and 88% for the 30‐, 90‐, and 180‐day follow‐up surveys, respectively. As shown in Table 1, participants average age was 52 years, and about half were women. The majority was non‐Hispanic black, and 21% preferred to complete the survey in Spanish. Diabetes, congestive heart failure, cancer, and chronic pulmonary disease were each prevalent in at least one‐fifth of our patient cohort. Demographic characteristics were similar between the 160 patients who completed all 3 follow‐up surveys and the 36 who missed at least 1 follow‐up survey. Among the latter group, 1 withdrew at 30 days, 1 withdrew and 4 had died at 90 days, and 1 withdrew and 9 had died at 180 days.

Figure 1 shows a timeline of the follow‐up surveys and utilization events in the form of overlapping histograms. The majority of 30‐day follow‐up questionnaires were completed earlier than targeted, at a median of 17 (interquartile range [IQR] 16, 20) days after discharge. Similarly, questionnaires targeted for 90 and 180 days were completed at medians of 78 (IQR 7684) and 167 (IQR 166169) days from discharge. Fifty‐four (28%) patients experienced a first utilization event in the first 30 days following discharge. During the 60‐, 90‐, and 30‐day intervals after the first, second, and third follow‐up surveys, respectively, 63 (33%), 54 (31%), and 16 (9%) patients experienced a first utilization event.

Figure 1

A significant improvement in MSAS physical symptom severity was detected between the hospitalization and the 30‐day follow‐up (Figure 2A). Although the mean Global Physical Health score was below the national mean of 50 at every survey period, a similar improvement in the measure was noted between the hospitalization and the 30‐day follow‐up (Figure 2B). The mean Global Mental Health score was also below the national mean but remained stable throughout the study (Figure 2C). The median GSRH was stable at 2 (IQR 23) at every survey wave (Figure 2D). Of note, compared to patients who completed all 3 follow‐up surveys, patients who missed at least 1 follow‐up reported higher MSAS score (1.5 vs 1.8, P=0.03), lower Global Physical Health (36.1 vs 33.5, P=0.09), and lower Global Mental Health (44.7 vs 41.0, P=0.03) during their hospitalization. In addition, patients with complete data experienced an average of 1.2 utilization events during the study, whereas those with missing data experienced an average of 2.1 utilization events (P=0.03).

Figure 2

The MSAS physical symptom severity and Global Physical Health scores from the index hospitalizations did not identify patients with a first utilization event within 30 days. However, patients with poor Global Mental Health and GSRH in the hospital were more likely to experience a utilization event within 14 days of discharge (Figure 3). During the postdischarge period, patients scoring poorly on each of the PRO measures trended toward a greater risk of an early utilization event, but the association between utilization and MSAS was most consistently significant (Figure 3A). In general, the associations with MSAS, Global Physical Health, and GSRH were stronger with the risk of utilization events within 14 days than within 30 days (Figure 3A,B,D). The Global Mental Health score was not associated with a subsequent utilization when measured during the 180‐day postdischarge period.

Figure 3

As shown in Table 2, Cox proportional hazard models incorporating covariates preserved most of the significant associations seen in the unadjusted analyses. Global Mental Health and good relative to poor GSRH obtained during the hospitalization remained significant. MSAS obtained at each postdischarge follow‐up trended positively with utilization and was statistically significant at 90 and 180 days. Global Physical Health obtained at each postdischarge follow‐up similarly trended negatively with utilization and was significant at 180 days. Each multivariate model incorporating a PRO measure with a significant coefficient contributed to better fit of the predictive model compared to the nested model without the PRO measure.

DISCUSSION

In this longitudinal cohort study, patients, on average, reported relatively severe symptoms, low PROMIS Global Physical and Mental Health scores, and poor GSRH during the inpatient stay in an urban safety‐net hospital. Symptom severity and Global Physical Health improved, on average, by 30 days before stabilizing, but their poor levels in the hospital did not predict 30‐day hospital‐based utilization events. On the other hand, Global Mental Health and GSRH were stable through hospitalizations, and patients scoring poorly on these measures were at greater risk of utilization events within 14 days of discharge. PRO measures obtained during the 180‐day postdischarge period trended toward distinguishing populations with greater baseline risk of proximate utilization events. However, MSAS physical symptom severity and Global Physical Health were more consistently predictive of these events at statistically significant levels compared to Global Mental Health and GSRH in our relatively small sample of patients. Each of these measures selectively improved the fit‐of‐risk prediction models for hospital‐based utilization.

Some of the heterogeneity in readmission risk is explained by differences in PRO measures. Although the MSAS score and Global Physical Health assessment were reliable predictors of utilization when measured in ambulatory settings, they were less discriminating during acute hospitalizations when everyone, on average, reported severe symptoms and poor function. Our results were consistent with other studies that demonstrated the fairly rapid recovery in symptoms that follow hospitalizations,[28, 29] and these measures may become informative of utilization risk as early as 2 weeks postdischarge. GSRH and Global Mental Health (a measure of health‐related quality of life) only predicted utilizations immediately at hospital discharge. As multidimensional measures that reflect physical, social, and emotional capacity, these measures may indicate vulnerabilities in patients least able to handle the stresses of the early postdischarge period.

There is growing momentum around collecting PRO measures in routine clinical care as quality indicators that capture patient‐centered concerns.[30] Our study explored a novel application of these measures whose routine collection will likely proliferate, not solely for the purpose of helping healthcare systems identify patients at risk of unplanned resource utilization. Although multidimensional PRO measures seldom reflect conditions directly modifiable by simple interventions, we believe that the association between physical symptom burden and utilization in our data reveals a possible target for practice improvement. Hospitalists have contributed enormously to shorter lengths of stay that risk sicker and quicker discharges.[31] To mitigate its potential side effects on symptom management, a discharge plan that acknowledges physical symptoms that sometimes persist or recur beyond the hospitalization may be appropriate. This may be accomplished by ensuring that acute symptoms are resolving, giving clear instructions for symptom management at home, as now the standard of care for conditions like asthma,[32] and explicitly communicating the presence of residual symptoms to providers entrusted with continuity care. As an effective feedback measure that can drive continuous quality improvement, we believe that a technology‐based surveillance strategy that spans both the inpatient and outpatient domains is necessary.[23]

There are some notable similarities and differences between the results of our study and a recent hospital‐based study of PRO measures that used data from the Multi‐Center Hospitalist Project.[16] The Physical Component Score of the SF12 is similar to the PROMIS Global Physical Health score in that both incorporate measures of physical function, perceived health, pain, and energy level. Curiously, the SF12 Physical Component Score, but not the PROMIS Global Physical Health score, was associated with 30‐day rehospitalizations. An important difference between the measures is where the SF12 asks about limitations during the past 4 weeks the PROMIS instrument inquires about physical function in general and levels of fatigue and pain in the past 7 days. Considering most hospitalizations last <7 days, the PROMIS instrument may better reflect the declines associated with the acute illness related to the hospitalization than the SF12 score. Additionally, the discrepancy between the association between hospital‐based GSRH and utilization in our study and the absence, thereof, in Hasan et al. is noteworthy. The difference here may be explained by their use of a 0‐ to 100‐point response scale in contrast to our study's verbally labeled 5‐point scale in the PROMIS instrument. The range of rating scales for survey questions is traditionally governed by the tension between the difficulty with mapping respondents judgment on an excessively large scale on one hand, and the failure of insufficient response options to discriminate between respondents with different underlying judgment on the other.[33] We suspect the former to be a drawback of the unlabeled 100‐point response scale, and conjecture that an association might be found in the Multi‐Center Hospitalist Study data if the responses were grouped into summative categories.

We recognize several limitations in our study. The first is the generalizability of our patient population to others, not insignificantly because of the high proportion of the uninsured (around 70% during the study period) and racial/ethnic minorities among them. Although utilization patterns are clearly affected by socioeconomic status,[34] there may also be differences in the way validated PRO measures are calibrated between patients of public and private healthcare systems.[35] Another limitation is our inability to count utilization events at institutions outside of the CCHHS during our study. However, because the study was conducted prior to Cook County's Medicaid expansion demonstration program as part of the Affordable Care Act,[36] many patients established in our system faced barriers to receiving nonemergency care outside of the CCHHS supporting our assumption that few of our patients were discharged from other hospitals. Causality cannot be established in observational studies. Consequently, high prior‐symptom burden may be associated with utilizations through unmeasured variables. Measures of symptom burden are vulnerable to overendorsement and amplification.[37, 38] Inferences based on statistical significance are affected by sample size, and our conclusions may change if conducted with a larger number of participants. Our response rates were excellent through the survey waves, but we did not achieve perfect follow‐up. Worse levels of PRO responses and higher levels of utilization among censored patients biased our results toward the null. Finally, although we did not find any predominant comorbidities associated with hospital‐based utilizations in our sample, our analyses may be vulnerable to inadequate control for illness severity, which may also have biased our results.

PRO measures are likely to be useful in clinical medicine.[39] But to fully apply the powers of PROs in informing clinically and operationally relevant outcomes, we must actively develop a system for obtaining these measures in routine clinical care. The availability of patient downtime makes hospitalizations conducive to gathering patient‐generated data, and may further enhance patient‐provider communication if survey output was readily available in electronic medical records. Exploring innovative strategies for collecting PROs in the hospital and beyond remains our future work.