High hospital occupancy is a fundamental challenge faced by healthcare systems in the United States.^1-3 However, few studies have examined the effect of high occupancy on outcomes in the inpatient setting,^4-9 and these showed mixed results. Hospital-acquired conditions (HACs), such as Clostridium difficile infection (CDI), are quality indicators for inpatient care and part of the Centers for Medicare and Medicaid Services’ Hospital-Acquired Conditions Reductions Program.^10-12 However, few studies—largely conducted outside of the US—have evaluated the association between inpatient occupancy and HACs. These studies showed increasing hospital-acquired infection rates with increasing occupancy.^13-15 Past studies of hospital occupancy have relied on annual average licensed bed counts, which are not a reliable measure of available and staffed beds and do not account for variations in patient volume and bed supply.¹⁶ Using a novel measure of inpatient occupancy, we tested the hypothesis that increasing inpatient occupancy is associated with a greater likelihood of CDI.

We performed a retrospective analysis of administrative data from non-federal, acute care hospitals in California during 2008–2012 using the Office of Statewide Health Planning and Development (OSHPD) Patient Discharge Data set, a complete census of all CA licensed general acute care hospital discharge records. This study was approved by the OSHPD Committee for the Protection of Human Subjects and was deemed exempt by our institution’s Institutional Review Board.

Selection of Participants

The study population consisted of fee-for-service Medicare enrollees ≥65 years admitted through the emergency department (ED) with a hospital length of stay (HLOS) <50 days and a primary discharge diagnosis of acute myocardial infarction (MI), pneumonia (PNA), or heart failure (HF; [identified through the respective Clinical Classification Software [CCS]).

The sample was restricted to discharges with a HLOS of <50 days, because those with longer HLOS (0.01% of study sample) were likely different in ways that may bias our findings (eg, they will likely be sicker). We limited our study to admissions through the ED to reduce potential selection bias by excluding elective admissions and hospital-to-hospital transfers, which are likely dependent on occupancy. MI, HF, and PNA diagnoses were selected because they are prevalent and have high inpatient mortality, allowing us to examine the effect of occupancy on some of the sickest inpatients.¹⁷

Hospital-acquired cases of CDI were identified as discharges (using ICD-9 code 008.45 for CDI) that were not marked as present-on-admission (POA) using the method described by Zhan et al.¹⁸ To avoid small facility outlying effects, we included hospitals that had 100 or more MI, HF, and PNA discharges that met the inclusion criteria over the study years.

OSHPD inpatient data were combined with OSHPD hospital annual financial data that contain hospital-level variables including ownership (City/County, District, Investor, and Non-Profit), geography (based on health services area), teaching status, urbanicity, and size based on the number of average annual licensed beds. If characteristics were not available for a given hospital for 1 or more years, the information from the closest available year was used for that hospital (replacement required for 10,504 (1.5%) cases; 4,856 otherwise eligible cases (0.7%) were dropped because the hospital was not included in the annual financial data for any year. Approximately 0.2% of records had invalid values for disposition, payer, or admission route, and were therefore dropped. Patient residence zip code-level socioeconomic status was measured using the percentage of families living below the poverty line, median family income, and the percentage of individuals with less than a high school degree among those aged ≥ 25 years¹⁹; these measures were divided into 3 groups (bottom quartile, top quartile, and middle 50%) for analysis.

Measure of Occupancy

Calculating Daily Census and Bed Capacity

We calculated the daily census using admission date and HLOS for each observation in our dataset. We approximated the bed capacity as the maximum daily census in the 121-day window (+/- 60 days) around each census day in each hospital. The 121-day window was chosen to increase the likelihood of capturing changes in bed availability (eg, due to unit closures) and seasonal variability. Our daily census does not include patients admitted with psychiatric and obstetrics diagnoses and long-term care/rehabilitation stays (identified through CCS categories and excluded) because these patients are not likely to compete for the same hospital resources as those receiving care for MI, HF, and PNA. See Appendix Table 1 for definition of the occupancy terms.

Calculating Relative Daily Occupancy

We developed a raw hospital-specific occupancy measure by dividing the daily census by the maximum census in each 121-day window for each hospital. We converted these raw measures to percentiles within the 121-day window to create a daily relative occupancy measure. For example, median level occupancy day would correspond to an occupancy of 0.5; a minimum or maximum occupancy day would correspond to 0 or 1, respectively. We preferred a relative occupancy measure because it assumes that what constitutes “high occupancy” likely depends on the usual occupancy level of the facility.

Measuring Admission Day Occupancy and Average Occupancy over Hospitalization

Using the relative daily occupancy values, we constructed patient-level variables representing occupancy on admission day and average occupancy during hospitalization.

Data Analysis

First, we estimated descriptive statistics of the sample for occupancy, patient-level (eg, age, race, gender, and severity of illness), hospital-level (eg, size, teaching status, and urbanicity), and incident-level (day-of-the-week and season) variables. Next, we used logistic regression with cluster standard errors to estimate the adjusted and unadjusted association of occupancy with CDI. For this analysis, occupancy was broken into 4 groups: 0.00-0.25 (low occupancy); 0.26-0.50; 0.51-0.75; and 0.76-1.00 (high occupancy), with the 0.0-0.25 group treated as the reference level. We fit separate models for admission and average occupancy and re-ran the latter model including HLOS as a sensitivity analysis.

Study Population and Hospitals

Across 327 hospitals, 558,829 discharges (including deaths) met our inclusion criteria and there were 2045 admissions with CDI. The hospital and discharge characteristics are reported in Appendix Table 2.

Relationship of Occupancy with CDI

With regard to admission occupancy, the 0.26-0.50 group did not have a significantly higher rate of CDI than the low occupancy group. Both the 0.51-0.75 and the 0.76-1.00 occupancy groups had 15% lower odds of CDI compared to the low occupancy group (Table). The adjusted results were similar, although the comparison between the low and high occupancy groups was marginally nonsignificant.

With regard to average occupancy, intermediate levels of occupancy (ie, 0.26-0.50 and 0.51-0.75 groups) had over 3-fold increased odds of CDI relative to the low occupancy group; the high occupancy group did not have significantly different odds of CDI compared to the low occupancy group (Table 1). The adjusted results were similar with no changes in statistical significance. Including HLOS tempered the adjusted odds of CDI to 1.6 for intermediate levels of occupancy, but these remained significantly higher than high or low occupancy.

Hospital occupancy is related to CDI. However, contrary to expectation, we found that higher admission and average occupancy over hospitalization were not related to more hospital-acquired CDI. CDI rates were highest for intermediate levels of average occupancy with lower CDI rates at high and low occupancy. CDI had an inverse relationship with admission occupancy.

These findings suggest that an exploration of the processes associated with hospitals accommodating higher occupancy might elucidate measures to reduce CDI. How do staffing, implementation of policies, and routine procedures vary when hospitals are busy or quiet? What aspects of care delivery that function well during high and low occupancy periods breakdown during intermediate occupancy? Hospital policies, practices, and procedures during different phases of occupancy might inform best practices. These data suggest that hospital occupancy level should be a routinely collected data element by infection control officers and that this should be linked with protocols triggered or modified with high or low occupancy that might affect HACs.

Previous studies in Europe found increasing hospital-acquired infection rates with increasing occupancy.^13-15 The authors postulated that increasing occupancy may limit available resources and increase nursing workloads, negatively impacting adherence to hand hygiene and cleaning protocols .⁸ However, these studies did not account for infections that were POA. In addition, our study examined hospitals in California after the 2006 implementation of the minimum nurse staffing policy, which means that staff to patient ratios could not fall below fixed thresholds that were typically higher than pre-policy ratios.¹⁹

This study had limitations pertaining to coded administrative data, including quality of coding and data validity. However, OSHPD has strict data reporting processes.²⁰ This study focused on 1 state; however, California is large with a demographically diverse population and hospital types, characteristics that would help generalize findings. Furthermore, when using the average occupancy measure, we could not determine whether the complication was acquired during the high occupancy period of the hospitalization.

Higher admission day occupancy was associated with lower likelihood of CDI, and CDI rates were lower at high and low average occupancy. These findings should prompt exploration of how hospitals react to occupancy changes and how those care processes translate into HACs in order to inform best practices for hospital care.

Acknowledgments

The authors would like to thank Ms. Amanda Kogowski, MPH and Mr. Rekar Taymour, MS for their editorial assistance with drafting the manuscript.

Disclosures

The authors have no conflicts to disclose.

Funding 

This study was funded by the National Institute on Aging.

High hospital occupancy is a fundamental challenge faced by healthcare systems in the United States.^1-3 However, few studies have examined the effect of high occupancy on outcomes in the inpatient setting,^4-9 and these showed mixed results. Hospital-acquired conditions (HACs), such as Clostridium difficile infection (CDI), are quality indicators for inpatient care and part of the Centers for Medicare and Medicaid Services’ Hospital-Acquired Conditions Reductions Program.^10-12 However, few studies—largely conducted outside of the US—have evaluated the association between inpatient occupancy and HACs. These studies showed increasing hospital-acquired infection rates with increasing occupancy.^13-15 Past studies of hospital occupancy have relied on annual average licensed bed counts, which are not a reliable measure of available and staffed beds and do not account for variations in patient volume and bed supply.¹⁶ Using a novel measure of inpatient occupancy, we tested the hypothesis that increasing inpatient occupancy is associated with a greater likelihood of CDI.

We performed a retrospective analysis of administrative data from non-federal, acute care hospitals in California during 2008–2012 using the Office of Statewide Health Planning and Development (OSHPD) Patient Discharge Data set, a complete census of all CA licensed general acute care hospital discharge records. This study was approved by the OSHPD Committee for the Protection of Human Subjects and was deemed exempt by our institution’s Institutional Review Board.

Selection of Participants

The study population consisted of fee-for-service Medicare enrollees ≥65 years admitted through the emergency department (ED) with a hospital length of stay (HLOS) <50 days and a primary discharge diagnosis of acute myocardial infarction (MI), pneumonia (PNA), or heart failure (HF; [identified through the respective Clinical Classification Software [CCS]).

The sample was restricted to discharges with a HLOS of <50 days, because those with longer HLOS (0.01% of study sample) were likely different in ways that may bias our findings (eg, they will likely be sicker). We limited our study to admissions through the ED to reduce potential selection bias by excluding elective admissions and hospital-to-hospital transfers, which are likely dependent on occupancy. MI, HF, and PNA diagnoses were selected because they are prevalent and have high inpatient mortality, allowing us to examine the effect of occupancy on some of the sickest inpatients.¹⁷

Hospital-acquired cases of CDI were identified as discharges (using ICD-9 code 008.45 for CDI) that were not marked as present-on-admission (POA) using the method described by Zhan et al.¹⁸ To avoid small facility outlying effects, we included hospitals that had 100 or more MI, HF, and PNA discharges that met the inclusion criteria over the study years.

OSHPD inpatient data were combined with OSHPD hospital annual financial data that contain hospital-level variables including ownership (City/County, District, Investor, and Non-Profit), geography (based on health services area), teaching status, urbanicity, and size based on the number of average annual licensed beds. If characteristics were not available for a given hospital for 1 or more years, the information from the closest available year was used for that hospital (replacement required for 10,504 (1.5%) cases; 4,856 otherwise eligible cases (0.7%) were dropped because the hospital was not included in the annual financial data for any year. Approximately 0.2% of records had invalid values for disposition, payer, or admission route, and were therefore dropped. Patient residence zip code-level socioeconomic status was measured using the percentage of families living below the poverty line, median family income, and the percentage of individuals with less than a high school degree among those aged ≥ 25 years¹⁹; these measures were divided into 3 groups (bottom quartile, top quartile, and middle 50%) for analysis.

Measure of Occupancy

Calculating Daily Census and Bed Capacity

We calculated the daily census using admission date and HLOS for each observation in our dataset. We approximated the bed capacity as the maximum daily census in the 121-day window (+/- 60 days) around each census day in each hospital. The 121-day window was chosen to increase the likelihood of capturing changes in bed availability (eg, due to unit closures) and seasonal variability. Our daily census does not include patients admitted with psychiatric and obstetrics diagnoses and long-term care/rehabilitation stays (identified through CCS categories and excluded) because these patients are not likely to compete for the same hospital resources as those receiving care for MI, HF, and PNA. See Appendix Table 1 for definition of the occupancy terms.

Calculating Relative Daily Occupancy

We developed a raw hospital-specific occupancy measure by dividing the daily census by the maximum census in each 121-day window for each hospital. We converted these raw measures to percentiles within the 121-day window to create a daily relative occupancy measure. For example, median level occupancy day would correspond to an occupancy of 0.5; a minimum or maximum occupancy day would correspond to 0 or 1, respectively. We preferred a relative occupancy measure because it assumes that what constitutes “high occupancy” likely depends on the usual occupancy level of the facility.

Measuring Admission Day Occupancy and Average Occupancy over Hospitalization

Using the relative daily occupancy values, we constructed patient-level variables representing occupancy on admission day and average occupancy during hospitalization.

Data Analysis

First, we estimated descriptive statistics of the sample for occupancy, patient-level (eg, age, race, gender, and severity of illness), hospital-level (eg, size, teaching status, and urbanicity), and incident-level (day-of-the-week and season) variables. Next, we used logistic regression with cluster standard errors to estimate the adjusted and unadjusted association of occupancy with CDI. For this analysis, occupancy was broken into 4 groups: 0.00-0.25 (low occupancy); 0.26-0.50; 0.51-0.75; and 0.76-1.00 (high occupancy), with the 0.0-0.25 group treated as the reference level. We fit separate models for admission and average occupancy and re-ran the latter model including HLOS as a sensitivity analysis.

Study Population and Hospitals

Across 327 hospitals, 558,829 discharges (including deaths) met our inclusion criteria and there were 2045 admissions with CDI. The hospital and discharge characteristics are reported in Appendix Table 2.

Relationship of Occupancy with CDI

With regard to admission occupancy, the 0.26-0.50 group did not have a significantly higher rate of CDI than the low occupancy group. Both the 0.51-0.75 and the 0.76-1.00 occupancy groups had 15% lower odds of CDI compared to the low occupancy group (Table). The adjusted results were similar, although the comparison between the low and high occupancy groups was marginally nonsignificant.

With regard to average occupancy, intermediate levels of occupancy (ie, 0.26-0.50 and 0.51-0.75 groups) had over 3-fold increased odds of CDI relative to the low occupancy group; the high occupancy group did not have significantly different odds of CDI compared to the low occupancy group (Table 1). The adjusted results were similar with no changes in statistical significance. Including HLOS tempered the adjusted odds of CDI to 1.6 for intermediate levels of occupancy, but these remained significantly higher than high or low occupancy.

Hospital occupancy is related to CDI. However, contrary to expectation, we found that higher admission and average occupancy over hospitalization were not related to more hospital-acquired CDI. CDI rates were highest for intermediate levels of average occupancy with lower CDI rates at high and low occupancy. CDI had an inverse relationship with admission occupancy.

These findings suggest that an exploration of the processes associated with hospitals accommodating higher occupancy might elucidate measures to reduce CDI. How do staffing, implementation of policies, and routine procedures vary when hospitals are busy or quiet? What aspects of care delivery that function well during high and low occupancy periods breakdown during intermediate occupancy? Hospital policies, practices, and procedures during different phases of occupancy might inform best practices. These data suggest that hospital occupancy level should be a routinely collected data element by infection control officers and that this should be linked with protocols triggered or modified with high or low occupancy that might affect HACs.

Previous studies in Europe found increasing hospital-acquired infection rates with increasing occupancy.^13-15 The authors postulated that increasing occupancy may limit available resources and increase nursing workloads, negatively impacting adherence to hand hygiene and cleaning protocols .⁸ However, these studies did not account for infections that were POA. In addition, our study examined hospitals in California after the 2006 implementation of the minimum nurse staffing policy, which means that staff to patient ratios could not fall below fixed thresholds that were typically higher than pre-policy ratios.¹⁹

This study had limitations pertaining to coded administrative data, including quality of coding and data validity. However, OSHPD has strict data reporting processes.²⁰ This study focused on 1 state; however, California is large with a demographically diverse population and hospital types, characteristics that would help generalize findings. Furthermore, when using the average occupancy measure, we could not determine whether the complication was acquired during the high occupancy period of the hospitalization.

Higher admission day occupancy was associated with lower likelihood of CDI, and CDI rates were lower at high and low average occupancy. These findings should prompt exploration of how hospitals react to occupancy changes and how those care processes translate into HACs in order to inform best practices for hospital care.

Acknowledgments

The authors would like to thank Ms. Amanda Kogowski, MPH and Mr. Rekar Taymour, MS for their editorial assistance with drafting the manuscript.

Disclosures

The authors have no conflicts to disclose.

Funding 

This study was funded by the National Institute on Aging.

High hospital occupancy is a fundamental challenge faced by healthcare systems in the United States.^1-3 However, few studies have examined the effect of high occupancy on outcomes in the inpatient setting,^4-9 and these showed mixed results. Hospital-acquired conditions (HACs), such as Clostridium difficile infection (CDI), are quality indicators for inpatient care and part of the Centers for Medicare and Medicaid Services’ Hospital-Acquired Conditions Reductions Program.^10-12 However, few studies—largely conducted outside of the US—have evaluated the association between inpatient occupancy and HACs. These studies showed increasing hospital-acquired infection rates with increasing occupancy.^13-15 Past studies of hospital occupancy have relied on annual average licensed bed counts, which are not a reliable measure of available and staffed beds and do not account for variations in patient volume and bed supply.¹⁶ Using a novel measure of inpatient occupancy, we tested the hypothesis that increasing inpatient occupancy is associated with a greater likelihood of CDI.

We performed a retrospective analysis of administrative data from non-federal, acute care hospitals in California during 2008–2012 using the Office of Statewide Health Planning and Development (OSHPD) Patient Discharge Data set, a complete census of all CA licensed general acute care hospital discharge records. This study was approved by the OSHPD Committee for the Protection of Human Subjects and was deemed exempt by our institution’s Institutional Review Board.

Selection of Participants

The study population consisted of fee-for-service Medicare enrollees ≥65 years admitted through the emergency department (ED) with a hospital length of stay (HLOS) <50 days and a primary discharge diagnosis of acute myocardial infarction (MI), pneumonia (PNA), or heart failure (HF; [identified through the respective Clinical Classification Software [CCS]).

The sample was restricted to discharges with a HLOS of <50 days, because those with longer HLOS (0.01% of study sample) were likely different in ways that may bias our findings (eg, they will likely be sicker). We limited our study to admissions through the ED to reduce potential selection bias by excluding elective admissions and hospital-to-hospital transfers, which are likely dependent on occupancy. MI, HF, and PNA diagnoses were selected because they are prevalent and have high inpatient mortality, allowing us to examine the effect of occupancy on some of the sickest inpatients.¹⁷

Hospital-acquired cases of CDI were identified as discharges (using ICD-9 code 008.45 for CDI) that were not marked as present-on-admission (POA) using the method described by Zhan et al.¹⁸ To avoid small facility outlying effects, we included hospitals that had 100 or more MI, HF, and PNA discharges that met the inclusion criteria over the study years.

OSHPD inpatient data were combined with OSHPD hospital annual financial data that contain hospital-level variables including ownership (City/County, District, Investor, and Non-Profit), geography (based on health services area), teaching status, urbanicity, and size based on the number of average annual licensed beds. If characteristics were not available for a given hospital for 1 or more years, the information from the closest available year was used for that hospital (replacement required for 10,504 (1.5%) cases; 4,856 otherwise eligible cases (0.7%) were dropped because the hospital was not included in the annual financial data for any year. Approximately 0.2% of records had invalid values for disposition, payer, or admission route, and were therefore dropped. Patient residence zip code-level socioeconomic status was measured using the percentage of families living below the poverty line, median family income, and the percentage of individuals with less than a high school degree among those aged ≥ 25 years¹⁹; these measures were divided into 3 groups (bottom quartile, top quartile, and middle 50%) for analysis.

Measure of Occupancy

Calculating Daily Census and Bed Capacity

We calculated the daily census using admission date and HLOS for each observation in our dataset. We approximated the bed capacity as the maximum daily census in the 121-day window (+/- 60 days) around each census day in each hospital. The 121-day window was chosen to increase the likelihood of capturing changes in bed availability (eg, due to unit closures) and seasonal variability. Our daily census does not include patients admitted with psychiatric and obstetrics diagnoses and long-term care/rehabilitation stays (identified through CCS categories and excluded) because these patients are not likely to compete for the same hospital resources as those receiving care for MI, HF, and PNA. See Appendix Table 1 for definition of the occupancy terms.

Calculating Relative Daily Occupancy

We developed a raw hospital-specific occupancy measure by dividing the daily census by the maximum census in each 121-day window for each hospital. We converted these raw measures to percentiles within the 121-day window to create a daily relative occupancy measure. For example, median level occupancy day would correspond to an occupancy of 0.5; a minimum or maximum occupancy day would correspond to 0 or 1, respectively. We preferred a relative occupancy measure because it assumes that what constitutes “high occupancy” likely depends on the usual occupancy level of the facility.

Measuring Admission Day Occupancy and Average Occupancy over Hospitalization

Using the relative daily occupancy values, we constructed patient-level variables representing occupancy on admission day and average occupancy during hospitalization.

Data Analysis

First, we estimated descriptive statistics of the sample for occupancy, patient-level (eg, age, race, gender, and severity of illness), hospital-level (eg, size, teaching status, and urbanicity), and incident-level (day-of-the-week and season) variables. Next, we used logistic regression with cluster standard errors to estimate the adjusted and unadjusted association of occupancy with CDI. For this analysis, occupancy was broken into 4 groups: 0.00-0.25 (low occupancy); 0.26-0.50; 0.51-0.75; and 0.76-1.00 (high occupancy), with the 0.0-0.25 group treated as the reference level. We fit separate models for admission and average occupancy and re-ran the latter model including HLOS as a sensitivity analysis.

Study Population and Hospitals

Across 327 hospitals, 558,829 discharges (including deaths) met our inclusion criteria and there were 2045 admissions with CDI. The hospital and discharge characteristics are reported in Appendix Table 2.

Relationship of Occupancy with CDI

With regard to admission occupancy, the 0.26-0.50 group did not have a significantly higher rate of CDI than the low occupancy group. Both the 0.51-0.75 and the 0.76-1.00 occupancy groups had 15% lower odds of CDI compared to the low occupancy group (Table). The adjusted results were similar, although the comparison between the low and high occupancy groups was marginally nonsignificant.

With regard to average occupancy, intermediate levels of occupancy (ie, 0.26-0.50 and 0.51-0.75 groups) had over 3-fold increased odds of CDI relative to the low occupancy group; the high occupancy group did not have significantly different odds of CDI compared to the low occupancy group (Table 1). The adjusted results were similar with no changes in statistical significance. Including HLOS tempered the adjusted odds of CDI to 1.6 for intermediate levels of occupancy, but these remained significantly higher than high or low occupancy.

Hospital occupancy is related to CDI. However, contrary to expectation, we found that higher admission and average occupancy over hospitalization were not related to more hospital-acquired CDI. CDI rates were highest for intermediate levels of average occupancy with lower CDI rates at high and low occupancy. CDI had an inverse relationship with admission occupancy.

These findings suggest that an exploration of the processes associated with hospitals accommodating higher occupancy might elucidate measures to reduce CDI. How do staffing, implementation of policies, and routine procedures vary when hospitals are busy or quiet? What aspects of care delivery that function well during high and low occupancy periods breakdown during intermediate occupancy? Hospital policies, practices, and procedures during different phases of occupancy might inform best practices. These data suggest that hospital occupancy level should be a routinely collected data element by infection control officers and that this should be linked with protocols triggered or modified with high or low occupancy that might affect HACs.

Previous studies in Europe found increasing hospital-acquired infection rates with increasing occupancy.^13-15 The authors postulated that increasing occupancy may limit available resources and increase nursing workloads, negatively impacting adherence to hand hygiene and cleaning protocols .⁸ However, these studies did not account for infections that were POA. In addition, our study examined hospitals in California after the 2006 implementation of the minimum nurse staffing policy, which means that staff to patient ratios could not fall below fixed thresholds that were typically higher than pre-policy ratios.¹⁹

This study had limitations pertaining to coded administrative data, including quality of coding and data validity. However, OSHPD has strict data reporting processes.²⁰ This study focused on 1 state; however, California is large with a demographically diverse population and hospital types, characteristics that would help generalize findings. Furthermore, when using the average occupancy measure, we could not determine whether the complication was acquired during the high occupancy period of the hospitalization.

Higher admission day occupancy was associated with lower likelihood of CDI, and CDI rates were lower at high and low average occupancy. These findings should prompt exploration of how hospitals react to occupancy changes and how those care processes translate into HACs in order to inform best practices for hospital care.

Acknowledgments

The authors would like to thank Ms. Amanda Kogowski, MPH and Mr. Rekar Taymour, MS for their editorial assistance with drafting the manuscript.

Disclosures

The authors have no conflicts to disclose.

Funding 

This study was funded by the National Institute on Aging.

Pediatric hospital medicine (PHM) has grown tremendously since Wachter first described the specialty in 1996.¹ Evidence of this growth is seen most markedly at the annual Pediatric Hospitalist Meeting, which has experienced an increase in attendance from 700 in 2013 to over 1,200 in 2017². Although the exact number of pediatric hospitalists in the United States is unknown, the American Academy of Pediatrics Section on Hospital Medicine (AAP SOHM) estimates that approximately 3,000-5,000 pediatric hospitalists currently practice in the country (personal communication).

As PHM programs have grown, variability has been reported in the roles, responsibilities, and workload among practitioners. Gosdin et al.³ reported large ranges and standard deviations in workload among full-time equivalents (FTEs) in academic PHM programs. However, this study’s ability to account for important nuances in program description was limited given that its data were obtained from an online survey.

Program variability, particularly regarding clinical hours and overall clinical burden (eg, in-house hours, census caps, and weekend coverage), is concerning given the well-reported increase in physician burn-out.^4,5 Benchmarking data regarding the overall workload of pediatric hospitalists can offer nationally recognized guidance to assist program leaders in building successful programs. With this goal in mind, we sought to obtain data on university-based PHM programs to describe the current average workload for a 1.0 clinical FTE pediatric hospitalist and to assess the perceptions of program directors regarding the sustainability of the current workload.

Study Design and Population

To obtain data with sufficient detail to compare programs, the authors, all of whom are practicing pediatric hospitalists at university-based programs, conducted structured interviews of PHM leaders in the United States. Given the absence of a single database for all PHM programs in the United States, the clinical division/program leaders of university-based programs were invited to participate through a post (with 2 reminders) to the AAP SOHM Listserv for PHM Division Leaders in May of 2017. To encourage participation, respondents were promised a summary of aggregate data. The study was exempted by the IRB of the University of Chicago.

Interview Content and Administration

The authors designed an 18-question structured interview regarding the current state of staffing in university-based PHM programs, with a focus on current descriptions of FTE, patient volume, and workload. Utilizing prior surveys³ as a basis, the authors iteratively determined the questions essential to understanding the programs’ current staffing models and ideal models. Considering the diversity of program models, interviews allowed for the clarification of questions and answers. A question regarding employment models was included to determine whether hospitalists were university-employed, hospital-employed, or a hybrid of the 2 modes of employment. The interview was also designed to establish a common language for work metrics (hours per year) for comparative purposes and to assess the perceived sustainability of the workload. Questions were provided in advance to provide respondents with sufficient time to collect data, thus increasing the accuracy of estimates. Respondents were asked, “Do you or your hospitalists have concerns about the sustainability of the model?” Sustainability was intentionally undefined to prevent limiting respondent perspective. For clarification, however, a follow-up comment that included examples was provided: “Faculty departures, reduction in total effort, and/or significant burn out.” The authors piloted the interview protocol by interviewing the division leaders of their own programs, and revisions were made based on feedback on feasibility and clarity. Finally, the AAP SOHM Subcommittee on Division Leaders provided feedback, which was incorporated.

Each author then interviewed 10-12 leaders (or designee) during May and June of 2017. Answers were recorded in REDCAP, an online survey and database tool that contains largely numeric data fields and has 1 field for narrative comments.

Data Analysis

Descriptive statistics were used to summarize interview responses, including median values with interquartile range. Data were compared between programs with models that were self-identified as either sustainable or unsustainable, with P-values in categorical variables from χ²-test or Fischer’s exact test and in continuous variables from Wilcoxon rank-sum test. 

Spearman correlation coefficient was used to evaluate the association between average protected time (defined as the percent of funded time for nonclinical roles) and percentage working full-time clinical effort. It was also used to evaluate hours per year per 1.0 FTE and total weekends per year per 1.0 FTE and perceived sustainability. Linear regression was used to determine whether associations differed between groups identifying as sustainable versus unsustainable.

Participation and Program Characteristics

Of the 143 subscribers to the listserv, which includes community and university-based programs, 62 division leaders/directors that self-identified by university-based hospitalist programs initially responded, and 56 completed phone interviews. Of these 56 respondents, 48% were university employed. The remainder were hospital employed (27%), had joint university/hospital appointments (13%), practiced in a private group (5%), or other models (7%).

Administration

A wide variation was reported in the clinical time expected of a 1.0 FTE hospitalist. Clinical time for 1.0 FTE was defined as the amount of clinical service a full-time hospitalist is expected to complete in 12 months (Table 1). The median hours worked per year were 1800 (Interquartile range [IQR] 1620,1975; mean 1796). The median number of weekends worked per year was 15.0 (IQR 12.5, 21; mean 16.8). Only 30% of pediatric hospitalists were full-time clinicians, whereas the rest had protected time for nonclinical duties. The average amount of protected time was 20% per full-time hospitalist.

Sustainability and Ideal FTE

Half of the division leaders reported that they or their hospitalists have concerns about the sustainability of the current workload. Programs perceived as sustainable required significantly fewer weekends per year (13 vs. 16, P < .02; Table 2) than those perceived as unsustainable. University-employed programs were more likely to be perceived as unsustainable (64% unsustainable vs. 32% unsustainable, P < .048), whereas programs with other employment models were more likely to be perceived as sustainable (Table 2). Total hours currently worked did not differ significantly between programs perceived as sustainable and unsustainable. Respondents reported an ideal workload for a 1.0 FTE of 1700 clinical hours (median). The hours worked per year for programs perceived as sustainable were statistically closer to their ideal than those perceived as unsustainable (P = .46; Table 2).

This study updates what has been previously reported about the structure and characteristics of university-based pediatric hospitalist programs.³ It also deepens our understanding of a relatively new field and the evolution of clinical coverage models. This evolution has been impacted by decreased resident work hours, increased patient complexity and acuity,⁶ and a broadened focus on care coordination and communication,⁷ while attempting to build and sustain a high-quality workforce.

This study is the first to use an interview-based method to determine the current PHM workload and to focus exclusively on university-based programs. Compared with the study by Gosdin et al,³ our study, which utilized interviews instead of surveys, was able to clarify questions and obtain workload data with a common language of hours per year. This approach allowed interviewees to incorporate subtleties, such as clinical vs. total FTE, in their responses. Our study found a slightly narrower range of clinical hours per year and extended the understanding of nonclinical duties by finding that university-based hospitalists have an average of 20% protected time from clinical duties.

In this study, we also explored the perceived sustainability of current clinical models and the ideal clinical model in hours per year. Half of respondents felt their current model was unsustainable. This result suggested that the field must continue to mitigate attrition and burnout.

Interestingly, the total number of clinical hours did not significantly differ in programs perceived to be unsustainable. Instead, a higher number of weekends worked and university employment were associated with lack of sustainability. We hypothesize that weekends have a disproportionate impact on work-life balance as compared with total hours, and that employment by a university may be a proxy for the increased academic and teaching demands of hospitalists without protected time. Future studies may better elucidate these findings and inform programmatic efforts to address sustainability.

Given that PHM is a relatively young field, considering the evolution of our clinical work model within the context of pediatric emergency medicine (PEM), a field that faces similar challenges in overnight and weekend staffing requirements, may be helpful. Gorelick et al.⁸ reported that total clinical work hours in PEM (combined academic and nonacademic programs) has decreased from 35.3 hours per week in 1998 to 26.7 in 2013. Extrapolating these numbers to an annual position with 5 weeks PTO/CME, the average PEM attending physician works 1254 clinical hours. These numbers demonstrate a marked difference compared with the average 1800 clinical work hours for PHM found in our study.

Although total hours trend lower in PEM, the authors noted continued challenges in sustainability with an estimated half of all PEM respondents indicating a plan to reduce hours or leave the field in the next 5 years and endorsing symptoms of burnout.⁶ These findings from PEM may motivate PHM leaders to be more aggressive in adjusting work models toward sustainability in the future.

Our study has several limitations. We utilized a convenience sampling approach that requires the voluntary participation of division directors. Although we had robust interest from respondents representing all major geographic areas, the respondent pool might conceivably over-represent those most interested in understanding and/or changing PHM clinical models. Overall, our sample size was smaller than that achieved by a survey approach. Nevertheless, this limitation was offset by controlling respondent type and clarifying questions, thus improving the quality of our obtained data.

This interview-based study of PHM directors describes the current state of clinical work models for university-based hospitalists. University-based PHM programs have similar mean and median total clinical hours per year. However, these hours are higher than those considered ideal by PHM directors, and many are concerned about the sustainability of current work models. Notably, programs that are university-employed or have higher weekends worked per year are more likely to be perceived as unsustainable. Future studies should explore differences between programs with sustainable work models and those with high levels of attrition and burnout.

Disclosures

The authors have no other conflicts to report.

Funding

A grant from the American Academy of Pediatrics Section on Hospital Medicine funded this study through the Subcommittee on Division and Program Leaders.

Pediatric hospital medicine (PHM) has grown tremendously since Wachter first described the specialty in 1996.¹ Evidence of this growth is seen most markedly at the annual Pediatric Hospitalist Meeting, which has experienced an increase in attendance from 700 in 2013 to over 1,200 in 2017². Although the exact number of pediatric hospitalists in the United States is unknown, the American Academy of Pediatrics Section on Hospital Medicine (AAP SOHM) estimates that approximately 3,000-5,000 pediatric hospitalists currently practice in the country (personal communication).

As PHM programs have grown, variability has been reported in the roles, responsibilities, and workload among practitioners. Gosdin et al.³ reported large ranges and standard deviations in workload among full-time equivalents (FTEs) in academic PHM programs. However, this study’s ability to account for important nuances in program description was limited given that its data were obtained from an online survey.

Program variability, particularly regarding clinical hours and overall clinical burden (eg, in-house hours, census caps, and weekend coverage), is concerning given the well-reported increase in physician burn-out.^4,5 Benchmarking data regarding the overall workload of pediatric hospitalists can offer nationally recognized guidance to assist program leaders in building successful programs. With this goal in mind, we sought to obtain data on university-based PHM programs to describe the current average workload for a 1.0 clinical FTE pediatric hospitalist and to assess the perceptions of program directors regarding the sustainability of the current workload.

Study Design and Population

To obtain data with sufficient detail to compare programs, the authors, all of whom are practicing pediatric hospitalists at university-based programs, conducted structured interviews of PHM leaders in the United States. Given the absence of a single database for all PHM programs in the United States, the clinical division/program leaders of university-based programs were invited to participate through a post (with 2 reminders) to the AAP SOHM Listserv for PHM Division Leaders in May of 2017. To encourage participation, respondents were promised a summary of aggregate data. The study was exempted by the IRB of the University of Chicago.

Interview Content and Administration

The authors designed an 18-question structured interview regarding the current state of staffing in university-based PHM programs, with a focus on current descriptions of FTE, patient volume, and workload. Utilizing prior surveys³ as a basis, the authors iteratively determined the questions essential to understanding the programs’ current staffing models and ideal models. Considering the diversity of program models, interviews allowed for the clarification of questions and answers. A question regarding employment models was included to determine whether hospitalists were university-employed, hospital-employed, or a hybrid of the 2 modes of employment. The interview was also designed to establish a common language for work metrics (hours per year) for comparative purposes and to assess the perceived sustainability of the workload. Questions were provided in advance to provide respondents with sufficient time to collect data, thus increasing the accuracy of estimates. Respondents were asked, “Do you or your hospitalists have concerns about the sustainability of the model?” Sustainability was intentionally undefined to prevent limiting respondent perspective. For clarification, however, a follow-up comment that included examples was provided: “Faculty departures, reduction in total effort, and/or significant burn out.” The authors piloted the interview protocol by interviewing the division leaders of their own programs, and revisions were made based on feedback on feasibility and clarity. Finally, the AAP SOHM Subcommittee on Division Leaders provided feedback, which was incorporated.

Each author then interviewed 10-12 leaders (or designee) during May and June of 2017. Answers were recorded in REDCAP, an online survey and database tool that contains largely numeric data fields and has 1 field for narrative comments.

Data Analysis

Descriptive statistics were used to summarize interview responses, including median values with interquartile range. Data were compared between programs with models that were self-identified as either sustainable or unsustainable, with P-values in categorical variables from χ²-test or Fischer’s exact test and in continuous variables from Wilcoxon rank-sum test. 

Spearman correlation coefficient was used to evaluate the association between average protected time (defined as the percent of funded time for nonclinical roles) and percentage working full-time clinical effort. It was also used to evaluate hours per year per 1.0 FTE and total weekends per year per 1.0 FTE and perceived sustainability. Linear regression was used to determine whether associations differed between groups identifying as sustainable versus unsustainable.

Participation and Program Characteristics

Of the 143 subscribers to the listserv, which includes community and university-based programs, 62 division leaders/directors that self-identified by university-based hospitalist programs initially responded, and 56 completed phone interviews. Of these 56 respondents, 48% were university employed. The remainder were hospital employed (27%), had joint university/hospital appointments (13%), practiced in a private group (5%), or other models (7%).

Administration

A wide variation was reported in the clinical time expected of a 1.0 FTE hospitalist. Clinical time for 1.0 FTE was defined as the amount of clinical service a full-time hospitalist is expected to complete in 12 months (Table 1). The median hours worked per year were 1800 (Interquartile range [IQR] 1620,1975; mean 1796). The median number of weekends worked per year was 15.0 (IQR 12.5, 21; mean 16.8). Only 30% of pediatric hospitalists were full-time clinicians, whereas the rest had protected time for nonclinical duties. The average amount of protected time was 20% per full-time hospitalist.

Sustainability and Ideal FTE

Half of the division leaders reported that they or their hospitalists have concerns about the sustainability of the current workload. Programs perceived as sustainable required significantly fewer weekends per year (13 vs. 16, P < .02; Table 2) than those perceived as unsustainable. University-employed programs were more likely to be perceived as unsustainable (64% unsustainable vs. 32% unsustainable, P < .048), whereas programs with other employment models were more likely to be perceived as sustainable (Table 2). Total hours currently worked did not differ significantly between programs perceived as sustainable and unsustainable. Respondents reported an ideal workload for a 1.0 FTE of 1700 clinical hours (median). The hours worked per year for programs perceived as sustainable were statistically closer to their ideal than those perceived as unsustainable (P = .46; Table 2).

This study updates what has been previously reported about the structure and characteristics of university-based pediatric hospitalist programs.³ It also deepens our understanding of a relatively new field and the evolution of clinical coverage models. This evolution has been impacted by decreased resident work hours, increased patient complexity and acuity,⁶ and a broadened focus on care coordination and communication,⁷ while attempting to build and sustain a high-quality workforce.

This study is the first to use an interview-based method to determine the current PHM workload and to focus exclusively on university-based programs. Compared with the study by Gosdin et al,³ our study, which utilized interviews instead of surveys, was able to clarify questions and obtain workload data with a common language of hours per year. This approach allowed interviewees to incorporate subtleties, such as clinical vs. total FTE, in their responses. Our study found a slightly narrower range of clinical hours per year and extended the understanding of nonclinical duties by finding that university-based hospitalists have an average of 20% protected time from clinical duties.

In this study, we also explored the perceived sustainability of current clinical models and the ideal clinical model in hours per year. Half of respondents felt their current model was unsustainable. This result suggested that the field must continue to mitigate attrition and burnout.

Interestingly, the total number of clinical hours did not significantly differ in programs perceived to be unsustainable. Instead, a higher number of weekends worked and university employment were associated with lack of sustainability. We hypothesize that weekends have a disproportionate impact on work-life balance as compared with total hours, and that employment by a university may be a proxy for the increased academic and teaching demands of hospitalists without protected time. Future studies may better elucidate these findings and inform programmatic efforts to address sustainability.

Given that PHM is a relatively young field, considering the evolution of our clinical work model within the context of pediatric emergency medicine (PEM), a field that faces similar challenges in overnight and weekend staffing requirements, may be helpful. Gorelick et al.⁸ reported that total clinical work hours in PEM (combined academic and nonacademic programs) has decreased from 35.3 hours per week in 1998 to 26.7 in 2013. Extrapolating these numbers to an annual position with 5 weeks PTO/CME, the average PEM attending physician works 1254 clinical hours. These numbers demonstrate a marked difference compared with the average 1800 clinical work hours for PHM found in our study.

Although total hours trend lower in PEM, the authors noted continued challenges in sustainability with an estimated half of all PEM respondents indicating a plan to reduce hours or leave the field in the next 5 years and endorsing symptoms of burnout.⁶ These findings from PEM may motivate PHM leaders to be more aggressive in adjusting work models toward sustainability in the future.

Our study has several limitations. We utilized a convenience sampling approach that requires the voluntary participation of division directors. Although we had robust interest from respondents representing all major geographic areas, the respondent pool might conceivably over-represent those most interested in understanding and/or changing PHM clinical models. Overall, our sample size was smaller than that achieved by a survey approach. Nevertheless, this limitation was offset by controlling respondent type and clarifying questions, thus improving the quality of our obtained data.

This interview-based study of PHM directors describes the current state of clinical work models for university-based hospitalists. University-based PHM programs have similar mean and median total clinical hours per year. However, these hours are higher than those considered ideal by PHM directors, and many are concerned about the sustainability of current work models. Notably, programs that are university-employed or have higher weekends worked per year are more likely to be perceived as unsustainable. Future studies should explore differences between programs with sustainable work models and those with high levels of attrition and burnout.

Disclosures

The authors have no other conflicts to report.

Funding

A grant from the American Academy of Pediatrics Section on Hospital Medicine funded this study through the Subcommittee on Division and Program Leaders.

Pediatric hospital medicine (PHM) has grown tremendously since Wachter first described the specialty in 1996.¹ Evidence of this growth is seen most markedly at the annual Pediatric Hospitalist Meeting, which has experienced an increase in attendance from 700 in 2013 to over 1,200 in 2017². Although the exact number of pediatric hospitalists in the United States is unknown, the American Academy of Pediatrics Section on Hospital Medicine (AAP SOHM) estimates that approximately 3,000-5,000 pediatric hospitalists currently practice in the country (personal communication).

As PHM programs have grown, variability has been reported in the roles, responsibilities, and workload among practitioners. Gosdin et al.³ reported large ranges and standard deviations in workload among full-time equivalents (FTEs) in academic PHM programs. However, this study’s ability to account for important nuances in program description was limited given that its data were obtained from an online survey.

Program variability, particularly regarding clinical hours and overall clinical burden (eg, in-house hours, census caps, and weekend coverage), is concerning given the well-reported increase in physician burn-out.^4,5 Benchmarking data regarding the overall workload of pediatric hospitalists can offer nationally recognized guidance to assist program leaders in building successful programs. With this goal in mind, we sought to obtain data on university-based PHM programs to describe the current average workload for a 1.0 clinical FTE pediatric hospitalist and to assess the perceptions of program directors regarding the sustainability of the current workload.

Study Design and Population

To obtain data with sufficient detail to compare programs, the authors, all of whom are practicing pediatric hospitalists at university-based programs, conducted structured interviews of PHM leaders in the United States. Given the absence of a single database for all PHM programs in the United States, the clinical division/program leaders of university-based programs were invited to participate through a post (with 2 reminders) to the AAP SOHM Listserv for PHM Division Leaders in May of 2017. To encourage participation, respondents were promised a summary of aggregate data. The study was exempted by the IRB of the University of Chicago.

Interview Content and Administration

The authors designed an 18-question structured interview regarding the current state of staffing in university-based PHM programs, with a focus on current descriptions of FTE, patient volume, and workload. Utilizing prior surveys³ as a basis, the authors iteratively determined the questions essential to understanding the programs’ current staffing models and ideal models. Considering the diversity of program models, interviews allowed for the clarification of questions and answers. A question regarding employment models was included to determine whether hospitalists were university-employed, hospital-employed, or a hybrid of the 2 modes of employment. The interview was also designed to establish a common language for work metrics (hours per year) for comparative purposes and to assess the perceived sustainability of the workload. Questions were provided in advance to provide respondents with sufficient time to collect data, thus increasing the accuracy of estimates. Respondents were asked, “Do you or your hospitalists have concerns about the sustainability of the model?” Sustainability was intentionally undefined to prevent limiting respondent perspective. For clarification, however, a follow-up comment that included examples was provided: “Faculty departures, reduction in total effort, and/or significant burn out.” The authors piloted the interview protocol by interviewing the division leaders of their own programs, and revisions were made based on feedback on feasibility and clarity. Finally, the AAP SOHM Subcommittee on Division Leaders provided feedback, which was incorporated.

Each author then interviewed 10-12 leaders (or designee) during May and June of 2017. Answers were recorded in REDCAP, an online survey and database tool that contains largely numeric data fields and has 1 field for narrative comments.

Data Analysis

Descriptive statistics were used to summarize interview responses, including median values with interquartile range. Data were compared between programs with models that were self-identified as either sustainable or unsustainable, with P-values in categorical variables from χ²-test or Fischer’s exact test and in continuous variables from Wilcoxon rank-sum test. 

Spearman correlation coefficient was used to evaluate the association between average protected time (defined as the percent of funded time for nonclinical roles) and percentage working full-time clinical effort. It was also used to evaluate hours per year per 1.0 FTE and total weekends per year per 1.0 FTE and perceived sustainability. Linear regression was used to determine whether associations differed between groups identifying as sustainable versus unsustainable.

Participation and Program Characteristics

Of the 143 subscribers to the listserv, which includes community and university-based programs, 62 division leaders/directors that self-identified by university-based hospitalist programs initially responded, and 56 completed phone interviews. Of these 56 respondents, 48% were university employed. The remainder were hospital employed (27%), had joint university/hospital appointments (13%), practiced in a private group (5%), or other models (7%).

Administration

A wide variation was reported in the clinical time expected of a 1.0 FTE hospitalist. Clinical time for 1.0 FTE was defined as the amount of clinical service a full-time hospitalist is expected to complete in 12 months (Table 1). The median hours worked per year were 1800 (Interquartile range [IQR] 1620,1975; mean 1796). The median number of weekends worked per year was 15.0 (IQR 12.5, 21; mean 16.8). Only 30% of pediatric hospitalists were full-time clinicians, whereas the rest had protected time for nonclinical duties. The average amount of protected time was 20% per full-time hospitalist.

Sustainability and Ideal FTE

Half of the division leaders reported that they or their hospitalists have concerns about the sustainability of the current workload. Programs perceived as sustainable required significantly fewer weekends per year (13 vs. 16, P < .02; Table 2) than those perceived as unsustainable. University-employed programs were more likely to be perceived as unsustainable (64% unsustainable vs. 32% unsustainable, P < .048), whereas programs with other employment models were more likely to be perceived as sustainable (Table 2). Total hours currently worked did not differ significantly between programs perceived as sustainable and unsustainable. Respondents reported an ideal workload for a 1.0 FTE of 1700 clinical hours (median). The hours worked per year for programs perceived as sustainable were statistically closer to their ideal than those perceived as unsustainable (P = .46; Table 2).

This study updates what has been previously reported about the structure and characteristics of university-based pediatric hospitalist programs.³ It also deepens our understanding of a relatively new field and the evolution of clinical coverage models. This evolution has been impacted by decreased resident work hours, increased patient complexity and acuity,⁶ and a broadened focus on care coordination and communication,⁷ while attempting to build and sustain a high-quality workforce.

This study is the first to use an interview-based method to determine the current PHM workload and to focus exclusively on university-based programs. Compared with the study by Gosdin et al,³ our study, which utilized interviews instead of surveys, was able to clarify questions and obtain workload data with a common language of hours per year. This approach allowed interviewees to incorporate subtleties, such as clinical vs. total FTE, in their responses. Our study found a slightly narrower range of clinical hours per year and extended the understanding of nonclinical duties by finding that university-based hospitalists have an average of 20% protected time from clinical duties.

In this study, we also explored the perceived sustainability of current clinical models and the ideal clinical model in hours per year. Half of respondents felt their current model was unsustainable. This result suggested that the field must continue to mitigate attrition and burnout.

Interestingly, the total number of clinical hours did not significantly differ in programs perceived to be unsustainable. Instead, a higher number of weekends worked and university employment were associated with lack of sustainability. We hypothesize that weekends have a disproportionate impact on work-life balance as compared with total hours, and that employment by a university may be a proxy for the increased academic and teaching demands of hospitalists without protected time. Future studies may better elucidate these findings and inform programmatic efforts to address sustainability.

Given that PHM is a relatively young field, considering the evolution of our clinical work model within the context of pediatric emergency medicine (PEM), a field that faces similar challenges in overnight and weekend staffing requirements, may be helpful. Gorelick et al.⁸ reported that total clinical work hours in PEM (combined academic and nonacademic programs) has decreased from 35.3 hours per week in 1998 to 26.7 in 2013. Extrapolating these numbers to an annual position with 5 weeks PTO/CME, the average PEM attending physician works 1254 clinical hours. These numbers demonstrate a marked difference compared with the average 1800 clinical work hours for PHM found in our study.

Although total hours trend lower in PEM, the authors noted continued challenges in sustainability with an estimated half of all PEM respondents indicating a plan to reduce hours or leave the field in the next 5 years and endorsing symptoms of burnout.⁶ These findings from PEM may motivate PHM leaders to be more aggressive in adjusting work models toward sustainability in the future.

Our study has several limitations. We utilized a convenience sampling approach that requires the voluntary participation of division directors. Although we had robust interest from respondents representing all major geographic areas, the respondent pool might conceivably over-represent those most interested in understanding and/or changing PHM clinical models. Overall, our sample size was smaller than that achieved by a survey approach. Nevertheless, this limitation was offset by controlling respondent type and clarifying questions, thus improving the quality of our obtained data.

This interview-based study of PHM directors describes the current state of clinical work models for university-based hospitalists. University-based PHM programs have similar mean and median total clinical hours per year. However, these hours are higher than those considered ideal by PHM directors, and many are concerned about the sustainability of current work models. Notably, programs that are university-employed or have higher weekends worked per year are more likely to be perceived as unsustainable. Future studies should explore differences between programs with sustainable work models and those with high levels of attrition and burnout.

Disclosures

The authors have no other conflicts to report.

Funding

A grant from the American Academy of Pediatrics Section on Hospital Medicine funded this study through the Subcommittee on Division and Program Leaders.

Hospital stays for pulmonary embolism (PE) represent a significant cost burden to the United States healthcare system.¹ The mean total hospitalization costs for treating a patient with PE ranges widely from $8,764 to $37,006, with an average reported length of stay between 4 and 5 days.^2,3 This cost range is attributed to many factors, including type of PE, therapy-induced bleeding risk requiring close monitoring, comorbidities, and social determinants of health. Given that patients with low-risk PE represent the majority of the cases, changes in approaches to care for this population can significantly impact the overall healthcare costs for PE. The European Society of Cardiology (ESC) guidelines incorporate well-validated risk scores, known as the pulmonary embolism severity index (PESI) and the simplified PESI (sPESI) score, and diagnostic test recommendations, including troponin test, echocardiography, and computed tomography, to evaluate patients with PE at varying risk for mortality.⁴ In these guidelines, the risk stratification algorithm for patients with a low PESI score or a sPESI score of zero does not include checking for the presence of troponin. In reality, practicing hospitalists frequently find that patients receiving a workup in the emergency department for suspected PE undergo troponin test. The ESC guidelines categorize patients with a low-risk score on PESI/sPESI, who subsequently have a positive troponin status, as intermediate low-risk and suggest consideration of hospitalization. The guidelines recommend patients with positive cardiac biomarkers to undergo assessment of right ventricular function through echocardiogram or computed tomography analysis. Moreover, the guidelines support early discharge or ambulatory treatment for low-risk patients who have a negative troponin status.⁴

The American College of Chest Physicians (ACCP) guidelines on venous thromboembolism (VTE) recommend that cardiac biomarkers should not be measured routinely in all patients with PE and that positive troponin status should discourage physicians from pursuing ambulatory treatment.⁵ Therefore, ambiguity lies within both guidelines with regard to how hospitalists should interpret a positive troponin status in patients with low risk, which in turn may lead to unnecessary hospitalizations and further imaging. This systematic review and meta-analysis aims to provide clarity, both about gaps in literature and about how practicing hospitalists should interpret troponins in patients with low-risk PE.

Data Sources and Searches

This systematic review and meta-analysis was performed in accordance with the established methods and Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) guidelines. We searched MEDLINE, SCOPUS, and Cochrane Controlled Trial Registry databases for studies published from inception to December 2016 by using the following key words: pulmonary embolism AND PESI OR “pulmonary embolism severity index.” Only articles written in English language were included. The full articles of potentially eligible studies were reviewed, and articles published only in abstract form were excluded.

Study Selection

Two investigators independently assessed the abstract of each article, and the full article was assessed if it fulfilled the following criteria: (1) the publication must be original; (2) inclusion of objectively diagnosed, hemodynamically stable patients (normotensive patients) with acute PE in the inpatient or outpatient setting; (3) inclusion of patients>19 years old; (4) use of the PESI or sPESI model to stratify patients into a low-risk group irrespective of any evidence of right ventricular dysfunction; and (5) testing of cardiac troponin levels (TnI-troponin I, TnT-troponin T, or hs-TnI/TnT-high sensitivity troponin I/T) in patients. Study design, sample size, duration of follow-up, type of troponin used, definition of hemodynamic stability, and specific type of outcome measured (endpoint) did not affect the study eligibility.

Data Extraction and Risk of Bias Assessment

Statistical Analysis

Data were summarized by using 30-day all-cause mortality only because it is the most consistent endpoint reported by all of the included studies. For each study, 30-day all-cause mortality was analyzed across the 2 troponin groups, and the results were summarized in terms of positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (PLR), negative likelihood ratio (NLR), and odds ratio (OR). To quantify the uncertainty in the LRs and ORs, we calculated 95% confidence intervals (CI).

Overall measures of PPV, NPV, PLR, and NLR were calculated on the pooled collection of data from the studies. LRs are one of the best measures of diagnostic accuracy; therefore, we defined the degree of probability of disease based on simple estimations that were reported by McGee.⁶ These estimations are independent of pretest probability and include the following: PLR 5.0 increases the probability of the outcome by about 30%, whereas NLR 0.20 decreases the probability of the outcome by 30%. To identify reasonable performance, we defined a PLR > 5 as an increase in moderate to high probability and a NLR < 0.20 as a decrease in moderate to high probability.⁶

The overall association between 30-day all-cause mortality and troponin classification among patients with low-risk PE was assessed using a mixed effects logistic regression model. The model included a random intercept to account for the correlation among the measurements for patients within a study. The exponentiated regression coefficient for troponin classification is the OR for 30-day all-cause mortality, comparing troponin-positive patients to troponin-negative patients. OR is reported with a 95% CI and a P value. A continuity correction (correction = 0.5) was applied to zero cells. Heterogeneity was measured using Cochran Q statistic and Higgins I² statistic.

Search Results

Figure 1 represents the PRISMA flow diagram for literature search and selection process to identify eligible studies for inclusion.

Study Characteristics

The abstracts of 117 articles were initially identified using the search strategy described above. Of these, 18 articles were deemed appropriate for review based on the criteria outlined in “Study Selection.” The full-text articles of the selected studies were obtained. Upon further evaluation, we identified 16 articles (Figure 1) eligible for the systematic review. Two studies were excluded because they did not provide the number of study participants that met the primary endpoints. The included studies were published from 2009–2016 (Table 1). For patients with low-risk PE, the number of patients with right ventricle dysfunction was either difficult to determine or not reported in all the studies.

Regarding study design, 11 studies were described as prospective cohorts and the remaining 5 studies were identified as retrospective (Table 1). Seven studies stratified participants’ risk of mortality by using sPESI, and 8 studies employed the PESI score. A total of 6952 participants diagnosed with PE were obtained, and 2662 (38%) were recognized as being low-risk based on either the PESI or sPESI. The sample sizes of the individual studies ranged from 121 to 1,291. The studies used either hs-cTnT, hs-cTnI, cTnT, cTnI, or a combination of hs-cTnT and cTnI or cTnT for troponin assay. Most studies used a pre-defined cut-off value to determine positive or negative troponin status.

Thirteen studies reported 30-day event rate as one of the primary endpoints. The 3 other studies included 90-day all-cause mortality, and 2 of them included in-hospital events. Secondary event rates were only reported in 4 studies and consisted of nonfatal PE, nonfatal major bleeding, and PE-related mortality.

Our systematic review revealed that 5 of the 16 studies used either hemodynamic decompensation, cardiopulmonary resuscitation, mechanical ventilation, or a combination of any of these parameters as part of their primary or secondary endpoint. However, none of the studies specified the number of patients that reached any of these endpoints. Furthermore, 10 of the 16 studies did not specify 30-day PE-related mortality outcomes. The most common endpoint was 30-day all-cause mortality, and only 7 studies reported outcomes with positive or negative troponin status.

Outcome Data of All Studies

A total of 2662 participants were categorized as being low risk based on the PESI or sPESI risk score. The pooled rate of PE-related mortality (specified and inferred) was 5 (0.46%) from 6 studies (1,093 patients), in which only 2 studies specified PE-related mortality as the primary endpoint (Vanni [2011]¹⁹ and Jimenez [2011]²⁰). The pooled rate of 30-day all-cause mortality was 24 (1.3%) from 12 studies (1882 patients). In 14 studies (2163 patients), the rates of recurrence of PE and major bleeding were 3 (0.14%) and 6 (0.28%), respectively.

Outcomes of Studies with Corresponding Troponin+ and Troponin –

Seven studies used positive or negative troponin status as endpoint to assess low-risk participants (Table 2). However, only 5 studies were included in the final meta-analysis because some data were missing in the Sanchez¹⁴ study and the Oszu⁸ study’s mortality endpoint was more than 30 days. The risk of bias within the studies was evaluated, and for most studies, the quality was of moderate degree (Supplementary Table 1). Table 2 shows the results for the overall pooled data stratified by study. In the pooled data, 463 (67%) patients tested negative for troponin and 228 (33%) tested positive. The overall mortality (from sensitivity analysis) including in-hospital, 30-day, and 90-day mortalities was 1.2%. The NPVs for all individual studies and the overall NPV are 1 or approximately 1. The overall PPVs and by study were low, ranging from 0 to 0.60. The PLRs and NLRs were not estimated for an outcome within an individual study if none of the patients experienced the outcome. When outcomes were only observed among troponin-negative patients, such as in the study of Moores (2009)²² who used 30-day all-cause mortality, the PLR had a value of zero. When outcomes were only observed among troponin-positive patients, as for 30-day all-cause mortality in the Hakemi(2015)⁹, Lauque (2014)¹⁰, and Lankeit(2011)¹⁶ studies, the NLR had a value of zero. For zero cells, a continuity correction of 0.5 was applied. The pooled likelihood ratios (LRs) for all-cause mortality were positive LR 2.04 (95% CI, 1.53 to 2.72) and negative LR 0.72 (95% CI, 0.37 to 1.40). The OR for all-cause mortality was 4.79 (95% CI 1.11 to 20.68, P = .0357).

A forest plot was created to visualize the PLR from each study included in the main analysis (Figure 2).

In this meta-analysis of 5 studies, which included 691 patients with low-risk PESI or sPESI scores, those tested positive for troponin had nearly a fivefold increased risk of 30-day all-cause mortality compared with patients who tested negative. However, the clinical significance of this association is unclear given that the CI is quite wide and mortality could be associated with PE versus other causes. Similar results were reported by other meta-analyses that consisted of patients with normotensive PE.^23-25 To our knowledge, the present meta-analysis is the first to report outcomes in patients with low-risk PE stratified by the presence of cardiac troponin.

A published paper on simplifying the clinical interpretation of LRs state that a positive LR of greater than 5 and a negative LR of less than 0.20 provide dependable evidence regarding reasonable prognostic performance.⁶ In our analysis, the positive LR was less than 5 and the negative LR’s CI included one. These results suggest a small statistical probability that a patient with a low PESI/sPESI score and a positive troponin status would benefit from inpatient monitoring; simultaneously, a negative troponin does not necessarily translate to safe outpatient therapy, based on our statistical analysis. Previous studies also reported nonextreme positive LRs.^23,24 We therefore conclude that low-risk PE patients with positive troponins may be eligible for safe ambulatory treatment or early discharge. However, the number of outcomes of interest (mortality) occurred in only 6 patients among the 228 patients who had positive troponin status. The majority of deaths were reported by Hakemi et al.⁹ in their retrospective cohort study; as such, drawing conclusions is difficult. Furthermore, the low 30-day all-cause mortality rate of 2.6% in the positive troponin group may have been affected by close monitoring of the patients, who commonly received hemodynamic and oxygen support. Based on these factors, our conclusion is relatively weak, and we cannot recommend a change in practice compared to existing guidelines. In general, additional prospective research is needed to determine whether patients with low-risk PE tested positive for troponin can receive care safely outside the hospital or, rather, require hospitalization similar to patients with intermediate-high risk PE.

We identified a number of other limitations in our analysis. First, aside from the relatively small number of pertinent studies in the literature, most of the studies are of low-moderate quality. Second, the troponin classification in various studies was not conducted using the same assay, and the cut-off value determining positive versus negative results in each case may have differed. These differences may have created some ambiguity or misclassification when the data were pooled together. Third, although the mixed effects logistic regression model controls for some of the variations among patients enrolled in different studies, significant differences exist in terms of patient characteristics or the protocol for follow-up care. This aspect was unaccounted for in this analysis. Lastly, pooled outcome events could not be retrieved from all of the included studies, which would have resulted in a misrepresentation of the true outcomes.

The ESC guidelines suggest avoiding cardiac biomarker testing in patients with low-risk PE because this practice does not have therapeutic implications. Moreover, ESC and ACCP guidelines both state that a positive cardiac biomarker should discourage treatment out of the hospital. The ACCP guidelines further encourage testing of cardiac biomarkers and/or evaluating right ventricular function via echocardiography when uncertainty exists regarding whether patients may require close in-hospital monitoring or not. Although no resounding evidence suggests that troponins have therapeutic implications in patients with low-risk PE, the current guidelines and our meta-analysis cannot offer an overwhelmingly convincing recommendation about whether or not patients with low-risk PE and positive cardiac biomarkers are best treated in the ambulatory or inpatient setting. Such patients may benefit from monitoring in an observation unit (eg, less than 24 or 48 hours), rather than requiring a full admission to the hospital. Nevertheless, our analysis shows that making this determination will require prospective studies that will utilize cardiac troponin status in predicting PE-related events, such as arrhythmia, acute respiratory failure, and hemodynamic decompensation, rather than all-cause mortality.

Until further studies, hospitalists should integrate the use of cardiac troponin and other clinical data, including those available from patient history, physical exam, and other laboratory testing, in determining whether or not to admit, observe, or discharge patients with low-risk PE. As the current guidelines recommend, we support consideration of right ventricular function assessment, via echocardiogram or computed tomography, in patients with positive cardiac troponins even when their PESI/sPESI score is low.

The authors would like to thank Megan Therese Smith, PhD and Lishi Zhang, MS for their contribution in providing a comprehensive statistical analysis of this meta-analysis.

Disclosures

The authors declare no conflicts of interest in the work under consideration for publication. Abdullah Mahayni and Mukti Patel, MD also declared no conflicts of interest with regard to the relevant financial activities outside the submitted work. Omar Darwish, DO and Alpesh Amin, MD also declared no relevant financial activities outside the submitted work; they are speakers for Bristol Myer Squibb and Pfizer regarding the anticoagulant, Apixaban, for treatment of venous thromboembolism and atrial fibrillation.

Hospital stays for pulmonary embolism (PE) represent a significant cost burden to the United States healthcare system.¹ The mean total hospitalization costs for treating a patient with PE ranges widely from $8,764 to $37,006, with an average reported length of stay between 4 and 5 days.^2,3 This cost range is attributed to many factors, including type of PE, therapy-induced bleeding risk requiring close monitoring, comorbidities, and social determinants of health. Given that patients with low-risk PE represent the majority of the cases, changes in approaches to care for this population can significantly impact the overall healthcare costs for PE. The European Society of Cardiology (ESC) guidelines incorporate well-validated risk scores, known as the pulmonary embolism severity index (PESI) and the simplified PESI (sPESI) score, and diagnostic test recommendations, including troponin test, echocardiography, and computed tomography, to evaluate patients with PE at varying risk for mortality.⁴ In these guidelines, the risk stratification algorithm for patients with a low PESI score or a sPESI score of zero does not include checking for the presence of troponin. In reality, practicing hospitalists frequently find that patients receiving a workup in the emergency department for suspected PE undergo troponin test. The ESC guidelines categorize patients with a low-risk score on PESI/sPESI, who subsequently have a positive troponin status, as intermediate low-risk and suggest consideration of hospitalization. The guidelines recommend patients with positive cardiac biomarkers to undergo assessment of right ventricular function through echocardiogram or computed tomography analysis. Moreover, the guidelines support early discharge or ambulatory treatment for low-risk patients who have a negative troponin status.⁴

The American College of Chest Physicians (ACCP) guidelines on venous thromboembolism (VTE) recommend that cardiac biomarkers should not be measured routinely in all patients with PE and that positive troponin status should discourage physicians from pursuing ambulatory treatment.⁵ Therefore, ambiguity lies within both guidelines with regard to how hospitalists should interpret a positive troponin status in patients with low risk, which in turn may lead to unnecessary hospitalizations and further imaging. This systematic review and meta-analysis aims to provide clarity, both about gaps in literature and about how practicing hospitalists should interpret troponins in patients with low-risk PE.

Data Sources and Searches

This systematic review and meta-analysis was performed in accordance with the established methods and Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) guidelines. We searched MEDLINE, SCOPUS, and Cochrane Controlled Trial Registry databases for studies published from inception to December 2016 by using the following key words: pulmonary embolism AND PESI OR “pulmonary embolism severity index.” Only articles written in English language were included. The full articles of potentially eligible studies were reviewed, and articles published only in abstract form were excluded.

Study Selection

Two investigators independently assessed the abstract of each article, and the full article was assessed if it fulfilled the following criteria: (1) the publication must be original; (2) inclusion of objectively diagnosed, hemodynamically stable patients (normotensive patients) with acute PE in the inpatient or outpatient setting; (3) inclusion of patients>19 years old; (4) use of the PESI or sPESI model to stratify patients into a low-risk group irrespective of any evidence of right ventricular dysfunction; and (5) testing of cardiac troponin levels (TnI-troponin I, TnT-troponin T, or hs-TnI/TnT-high sensitivity troponin I/T) in patients. Study design, sample size, duration of follow-up, type of troponin used, definition of hemodynamic stability, and specific type of outcome measured (endpoint) did not affect the study eligibility.

Data Extraction and Risk of Bias Assessment

Statistical Analysis

Data were summarized by using 30-day all-cause mortality only because it is the most consistent endpoint reported by all of the included studies. For each study, 30-day all-cause mortality was analyzed across the 2 troponin groups, and the results were summarized in terms of positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (PLR), negative likelihood ratio (NLR), and odds ratio (OR). To quantify the uncertainty in the LRs and ORs, we calculated 95% confidence intervals (CI).

Overall measures of PPV, NPV, PLR, and NLR were calculated on the pooled collection of data from the studies. LRs are one of the best measures of diagnostic accuracy; therefore, we defined the degree of probability of disease based on simple estimations that were reported by McGee.⁶ These estimations are independent of pretest probability and include the following: PLR 5.0 increases the probability of the outcome by about 30%, whereas NLR 0.20 decreases the probability of the outcome by 30%. To identify reasonable performance, we defined a PLR > 5 as an increase in moderate to high probability and a NLR < 0.20 as a decrease in moderate to high probability.⁶

The overall association between 30-day all-cause mortality and troponin classification among patients with low-risk PE was assessed using a mixed effects logistic regression model. The model included a random intercept to account for the correlation among the measurements for patients within a study. The exponentiated regression coefficient for troponin classification is the OR for 30-day all-cause mortality, comparing troponin-positive patients to troponin-negative patients. OR is reported with a 95% CI and a P value. A continuity correction (correction = 0.5) was applied to zero cells. Heterogeneity was measured using Cochran Q statistic and Higgins I² statistic.

Search Results

Figure 1 represents the PRISMA flow diagram for literature search and selection process to identify eligible studies for inclusion.

Study Characteristics

The abstracts of 117 articles were initially identified using the search strategy described above. Of these, 18 articles were deemed appropriate for review based on the criteria outlined in “Study Selection.” The full-text articles of the selected studies were obtained. Upon further evaluation, we identified 16 articles (Figure 1) eligible for the systematic review. Two studies were excluded because they did not provide the number of study participants that met the primary endpoints. The included studies were published from 2009–2016 (Table 1). For patients with low-risk PE, the number of patients with right ventricle dysfunction was either difficult to determine or not reported in all the studies.

Regarding study design, 11 studies were described as prospective cohorts and the remaining 5 studies were identified as retrospective (Table 1). Seven studies stratified participants’ risk of mortality by using sPESI, and 8 studies employed the PESI score. A total of 6952 participants diagnosed with PE were obtained, and 2662 (38%) were recognized as being low-risk based on either the PESI or sPESI. The sample sizes of the individual studies ranged from 121 to 1,291. The studies used either hs-cTnT, hs-cTnI, cTnT, cTnI, or a combination of hs-cTnT and cTnI or cTnT for troponin assay. Most studies used a pre-defined cut-off value to determine positive or negative troponin status.

Thirteen studies reported 30-day event rate as one of the primary endpoints. The 3 other studies included 90-day all-cause mortality, and 2 of them included in-hospital events. Secondary event rates were only reported in 4 studies and consisted of nonfatal PE, nonfatal major bleeding, and PE-related mortality.

Our systematic review revealed that 5 of the 16 studies used either hemodynamic decompensation, cardiopulmonary resuscitation, mechanical ventilation, or a combination of any of these parameters as part of their primary or secondary endpoint. However, none of the studies specified the number of patients that reached any of these endpoints. Furthermore, 10 of the 16 studies did not specify 30-day PE-related mortality outcomes. The most common endpoint was 30-day all-cause mortality, and only 7 studies reported outcomes with positive or negative troponin status.

Outcome Data of All Studies

A total of 2662 participants were categorized as being low risk based on the PESI or sPESI risk score. The pooled rate of PE-related mortality (specified and inferred) was 5 (0.46%) from 6 studies (1,093 patients), in which only 2 studies specified PE-related mortality as the primary endpoint (Vanni [2011]¹⁹ and Jimenez [2011]²⁰). The pooled rate of 30-day all-cause mortality was 24 (1.3%) from 12 studies (1882 patients). In 14 studies (2163 patients), the rates of recurrence of PE and major bleeding were 3 (0.14%) and 6 (0.28%), respectively.

Outcomes of Studies with Corresponding Troponin+ and Troponin –

Seven studies used positive or negative troponin status as endpoint to assess low-risk participants (Table 2). However, only 5 studies were included in the final meta-analysis because some data were missing in the Sanchez¹⁴ study and the Oszu⁸ study’s mortality endpoint was more than 30 days. The risk of bias within the studies was evaluated, and for most studies, the quality was of moderate degree (Supplementary Table 1). Table 2 shows the results for the overall pooled data stratified by study. In the pooled data, 463 (67%) patients tested negative for troponin and 228 (33%) tested positive. The overall mortality (from sensitivity analysis) including in-hospital, 30-day, and 90-day mortalities was 1.2%. The NPVs for all individual studies and the overall NPV are 1 or approximately 1. The overall PPVs and by study were low, ranging from 0 to 0.60. The PLRs and NLRs were not estimated for an outcome within an individual study if none of the patients experienced the outcome. When outcomes were only observed among troponin-negative patients, such as in the study of Moores (2009)²² who used 30-day all-cause mortality, the PLR had a value of zero. When outcomes were only observed among troponin-positive patients, as for 30-day all-cause mortality in the Hakemi(2015)⁹, Lauque (2014)¹⁰, and Lankeit(2011)¹⁶ studies, the NLR had a value of zero. For zero cells, a continuity correction of 0.5 was applied. The pooled likelihood ratios (LRs) for all-cause mortality were positive LR 2.04 (95% CI, 1.53 to 2.72) and negative LR 0.72 (95% CI, 0.37 to 1.40). The OR for all-cause mortality was 4.79 (95% CI 1.11 to 20.68, P = .0357).

A forest plot was created to visualize the PLR from each study included in the main analysis (Figure 2).

In this meta-analysis of 5 studies, which included 691 patients with low-risk PESI or sPESI scores, those tested positive for troponin had nearly a fivefold increased risk of 30-day all-cause mortality compared with patients who tested negative. However, the clinical significance of this association is unclear given that the CI is quite wide and mortality could be associated with PE versus other causes. Similar results were reported by other meta-analyses that consisted of patients with normotensive PE.^23-25 To our knowledge, the present meta-analysis is the first to report outcomes in patients with low-risk PE stratified by the presence of cardiac troponin.

A published paper on simplifying the clinical interpretation of LRs state that a positive LR of greater than 5 and a negative LR of less than 0.20 provide dependable evidence regarding reasonable prognostic performance.⁶ In our analysis, the positive LR was less than 5 and the negative LR’s CI included one. These results suggest a small statistical probability that a patient with a low PESI/sPESI score and a positive troponin status would benefit from inpatient monitoring; simultaneously, a negative troponin does not necessarily translate to safe outpatient therapy, based on our statistical analysis. Previous studies also reported nonextreme positive LRs.^23,24 We therefore conclude that low-risk PE patients with positive troponins may be eligible for safe ambulatory treatment or early discharge. However, the number of outcomes of interest (mortality) occurred in only 6 patients among the 228 patients who had positive troponin status. The majority of deaths were reported by Hakemi et al.⁹ in their retrospective cohort study; as such, drawing conclusions is difficult. Furthermore, the low 30-day all-cause mortality rate of 2.6% in the positive troponin group may have been affected by close monitoring of the patients, who commonly received hemodynamic and oxygen support. Based on these factors, our conclusion is relatively weak, and we cannot recommend a change in practice compared to existing guidelines. In general, additional prospective research is needed to determine whether patients with low-risk PE tested positive for troponin can receive care safely outside the hospital or, rather, require hospitalization similar to patients with intermediate-high risk PE.

We identified a number of other limitations in our analysis. First, aside from the relatively small number of pertinent studies in the literature, most of the studies are of low-moderate quality. Second, the troponin classification in various studies was not conducted using the same assay, and the cut-off value determining positive versus negative results in each case may have differed. These differences may have created some ambiguity or misclassification when the data were pooled together. Third, although the mixed effects logistic regression model controls for some of the variations among patients enrolled in different studies, significant differences exist in terms of patient characteristics or the protocol for follow-up care. This aspect was unaccounted for in this analysis. Lastly, pooled outcome events could not be retrieved from all of the included studies, which would have resulted in a misrepresentation of the true outcomes.

The ESC guidelines suggest avoiding cardiac biomarker testing in patients with low-risk PE because this practice does not have therapeutic implications. Moreover, ESC and ACCP guidelines both state that a positive cardiac biomarker should discourage treatment out of the hospital. The ACCP guidelines further encourage testing of cardiac biomarkers and/or evaluating right ventricular function via echocardiography when uncertainty exists regarding whether patients may require close in-hospital monitoring or not. Although no resounding evidence suggests that troponins have therapeutic implications in patients with low-risk PE, the current guidelines and our meta-analysis cannot offer an overwhelmingly convincing recommendation about whether or not patients with low-risk PE and positive cardiac biomarkers are best treated in the ambulatory or inpatient setting. Such patients may benefit from monitoring in an observation unit (eg, less than 24 or 48 hours), rather than requiring a full admission to the hospital. Nevertheless, our analysis shows that making this determination will require prospective studies that will utilize cardiac troponin status in predicting PE-related events, such as arrhythmia, acute respiratory failure, and hemodynamic decompensation, rather than all-cause mortality.

Until further studies, hospitalists should integrate the use of cardiac troponin and other clinical data, including those available from patient history, physical exam, and other laboratory testing, in determining whether or not to admit, observe, or discharge patients with low-risk PE. As the current guidelines recommend, we support consideration of right ventricular function assessment, via echocardiogram or computed tomography, in patients with positive cardiac troponins even when their PESI/sPESI score is low.

The authors would like to thank Megan Therese Smith, PhD and Lishi Zhang, MS for their contribution in providing a comprehensive statistical analysis of this meta-analysis.

Disclosures

The authors declare no conflicts of interest in the work under consideration for publication. Abdullah Mahayni and Mukti Patel, MD also declared no conflicts of interest with regard to the relevant financial activities outside the submitted work. Omar Darwish, DO and Alpesh Amin, MD also declared no relevant financial activities outside the submitted work; they are speakers for Bristol Myer Squibb and Pfizer regarding the anticoagulant, Apixaban, for treatment of venous thromboembolism and atrial fibrillation.

Hospital stays for pulmonary embolism (PE) represent a significant cost burden to the United States healthcare system.¹ The mean total hospitalization costs for treating a patient with PE ranges widely from $8,764 to $37,006, with an average reported length of stay between 4 and 5 days.^2,3 This cost range is attributed to many factors, including type of PE, therapy-induced bleeding risk requiring close monitoring, comorbidities, and social determinants of health. Given that patients with low-risk PE represent the majority of the cases, changes in approaches to care for this population can significantly impact the overall healthcare costs for PE. The European Society of Cardiology (ESC) guidelines incorporate well-validated risk scores, known as the pulmonary embolism severity index (PESI) and the simplified PESI (sPESI) score, and diagnostic test recommendations, including troponin test, echocardiography, and computed tomography, to evaluate patients with PE at varying risk for mortality.⁴ In these guidelines, the risk stratification algorithm for patients with a low PESI score or a sPESI score of zero does not include checking for the presence of troponin. In reality, practicing hospitalists frequently find that patients receiving a workup in the emergency department for suspected PE undergo troponin test. The ESC guidelines categorize patients with a low-risk score on PESI/sPESI, who subsequently have a positive troponin status, as intermediate low-risk and suggest consideration of hospitalization. The guidelines recommend patients with positive cardiac biomarkers to undergo assessment of right ventricular function through echocardiogram or computed tomography analysis. Moreover, the guidelines support early discharge or ambulatory treatment for low-risk patients who have a negative troponin status.⁴

The American College of Chest Physicians (ACCP) guidelines on venous thromboembolism (VTE) recommend that cardiac biomarkers should not be measured routinely in all patients with PE and that positive troponin status should discourage physicians from pursuing ambulatory treatment.⁵ Therefore, ambiguity lies within both guidelines with regard to how hospitalists should interpret a positive troponin status in patients with low risk, which in turn may lead to unnecessary hospitalizations and further imaging. This systematic review and meta-analysis aims to provide clarity, both about gaps in literature and about how practicing hospitalists should interpret troponins in patients with low-risk PE.

Data Sources and Searches

This systematic review and meta-analysis was performed in accordance with the established methods and Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) guidelines. We searched MEDLINE, SCOPUS, and Cochrane Controlled Trial Registry databases for studies published from inception to December 2016 by using the following key words: pulmonary embolism AND PESI OR “pulmonary embolism severity index.” Only articles written in English language were included. The full articles of potentially eligible studies were reviewed, and articles published only in abstract form were excluded.

Study Selection

Two investigators independently assessed the abstract of each article, and the full article was assessed if it fulfilled the following criteria: (1) the publication must be original; (2) inclusion of objectively diagnosed, hemodynamically stable patients (normotensive patients) with acute PE in the inpatient or outpatient setting; (3) inclusion of patients>19 years old; (4) use of the PESI or sPESI model to stratify patients into a low-risk group irrespective of any evidence of right ventricular dysfunction; and (5) testing of cardiac troponin levels (TnI-troponin I, TnT-troponin T, or hs-TnI/TnT-high sensitivity troponin I/T) in patients. Study design, sample size, duration of follow-up, type of troponin used, definition of hemodynamic stability, and specific type of outcome measured (endpoint) did not affect the study eligibility.

Data Extraction and Risk of Bias Assessment

Statistical Analysis

Data were summarized by using 30-day all-cause mortality only because it is the most consistent endpoint reported by all of the included studies. For each study, 30-day all-cause mortality was analyzed across the 2 troponin groups, and the results were summarized in terms of positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (PLR), negative likelihood ratio (NLR), and odds ratio (OR). To quantify the uncertainty in the LRs and ORs, we calculated 95% confidence intervals (CI).

Overall measures of PPV, NPV, PLR, and NLR were calculated on the pooled collection of data from the studies. LRs are one of the best measures of diagnostic accuracy; therefore, we defined the degree of probability of disease based on simple estimations that were reported by McGee.⁶ These estimations are independent of pretest probability and include the following: PLR 5.0 increases the probability of the outcome by about 30%, whereas NLR 0.20 decreases the probability of the outcome by 30%. To identify reasonable performance, we defined a PLR > 5 as an increase in moderate to high probability and a NLR < 0.20 as a decrease in moderate to high probability.⁶

The overall association between 30-day all-cause mortality and troponin classification among patients with low-risk PE was assessed using a mixed effects logistic regression model. The model included a random intercept to account for the correlation among the measurements for patients within a study. The exponentiated regression coefficient for troponin classification is the OR for 30-day all-cause mortality, comparing troponin-positive patients to troponin-negative patients. OR is reported with a 95% CI and a P value. A continuity correction (correction = 0.5) was applied to zero cells. Heterogeneity was measured using Cochran Q statistic and Higgins I² statistic.

Search Results

Figure 1 represents the PRISMA flow diagram for literature search and selection process to identify eligible studies for inclusion.

Study Characteristics

The abstracts of 117 articles were initially identified using the search strategy described above. Of these, 18 articles were deemed appropriate for review based on the criteria outlined in “Study Selection.” The full-text articles of the selected studies were obtained. Upon further evaluation, we identified 16 articles (Figure 1) eligible for the systematic review. Two studies were excluded because they did not provide the number of study participants that met the primary endpoints. The included studies were published from 2009–2016 (Table 1). For patients with low-risk PE, the number of patients with right ventricle dysfunction was either difficult to determine or not reported in all the studies.

Regarding study design, 11 studies were described as prospective cohorts and the remaining 5 studies were identified as retrospective (Table 1). Seven studies stratified participants’ risk of mortality by using sPESI, and 8 studies employed the PESI score. A total of 6952 participants diagnosed with PE were obtained, and 2662 (38%) were recognized as being low-risk based on either the PESI or sPESI. The sample sizes of the individual studies ranged from 121 to 1,291. The studies used either hs-cTnT, hs-cTnI, cTnT, cTnI, or a combination of hs-cTnT and cTnI or cTnT for troponin assay. Most studies used a pre-defined cut-off value to determine positive or negative troponin status.

Thirteen studies reported 30-day event rate as one of the primary endpoints. The 3 other studies included 90-day all-cause mortality, and 2 of them included in-hospital events. Secondary event rates were only reported in 4 studies and consisted of nonfatal PE, nonfatal major bleeding, and PE-related mortality.

Our systematic review revealed that 5 of the 16 studies used either hemodynamic decompensation, cardiopulmonary resuscitation, mechanical ventilation, or a combination of any of these parameters as part of their primary or secondary endpoint. However, none of the studies specified the number of patients that reached any of these endpoints. Furthermore, 10 of the 16 studies did not specify 30-day PE-related mortality outcomes. The most common endpoint was 30-day all-cause mortality, and only 7 studies reported outcomes with positive or negative troponin status.

Outcome Data of All Studies

A total of 2662 participants were categorized as being low risk based on the PESI or sPESI risk score. The pooled rate of PE-related mortality (specified and inferred) was 5 (0.46%) from 6 studies (1,093 patients), in which only 2 studies specified PE-related mortality as the primary endpoint (Vanni [2011]¹⁹ and Jimenez [2011]²⁰). The pooled rate of 30-day all-cause mortality was 24 (1.3%) from 12 studies (1882 patients). In 14 studies (2163 patients), the rates of recurrence of PE and major bleeding were 3 (0.14%) and 6 (0.28%), respectively.

Outcomes of Studies with Corresponding Troponin+ and Troponin –

Seven studies used positive or negative troponin status as endpoint to assess low-risk participants (Table 2). However, only 5 studies were included in the final meta-analysis because some data were missing in the Sanchez¹⁴ study and the Oszu⁸ study’s mortality endpoint was more than 30 days. The risk of bias within the studies was evaluated, and for most studies, the quality was of moderate degree (Supplementary Table 1). Table 2 shows the results for the overall pooled data stratified by study. In the pooled data, 463 (67%) patients tested negative for troponin and 228 (33%) tested positive. The overall mortality (from sensitivity analysis) including in-hospital, 30-day, and 90-day mortalities was 1.2%. The NPVs for all individual studies and the overall NPV are 1 or approximately 1. The overall PPVs and by study were low, ranging from 0 to 0.60. The PLRs and NLRs were not estimated for an outcome within an individual study if none of the patients experienced the outcome. When outcomes were only observed among troponin-negative patients, such as in the study of Moores (2009)²² who used 30-day all-cause mortality, the PLR had a value of zero. When outcomes were only observed among troponin-positive patients, as for 30-day all-cause mortality in the Hakemi(2015)⁹, Lauque (2014)¹⁰, and Lankeit(2011)¹⁶ studies, the NLR had a value of zero. For zero cells, a continuity correction of 0.5 was applied. The pooled likelihood ratios (LRs) for all-cause mortality were positive LR 2.04 (95% CI, 1.53 to 2.72) and negative LR 0.72 (95% CI, 0.37 to 1.40). The OR for all-cause mortality was 4.79 (95% CI 1.11 to 20.68, P = .0357).

A forest plot was created to visualize the PLR from each study included in the main analysis (Figure 2).

In this meta-analysis of 5 studies, which included 691 patients with low-risk PESI or sPESI scores, those tested positive for troponin had nearly a fivefold increased risk of 30-day all-cause mortality compared with patients who tested negative. However, the clinical significance of this association is unclear given that the CI is quite wide and mortality could be associated with PE versus other causes. Similar results were reported by other meta-analyses that consisted of patients with normotensive PE.^23-25 To our knowledge, the present meta-analysis is the first to report outcomes in patients with low-risk PE stratified by the presence of cardiac troponin.

A published paper on simplifying the clinical interpretation of LRs state that a positive LR of greater than 5 and a negative LR of less than 0.20 provide dependable evidence regarding reasonable prognostic performance.⁶ In our analysis, the positive LR was less than 5 and the negative LR’s CI included one. These results suggest a small statistical probability that a patient with a low PESI/sPESI score and a positive troponin status would benefit from inpatient monitoring; simultaneously, a negative troponin does not necessarily translate to safe outpatient therapy, based on our statistical analysis. Previous studies also reported nonextreme positive LRs.^23,24 We therefore conclude that low-risk PE patients with positive troponins may be eligible for safe ambulatory treatment or early discharge. However, the number of outcomes of interest (mortality) occurred in only 6 patients among the 228 patients who had positive troponin status. The majority of deaths were reported by Hakemi et al.⁹ in their retrospective cohort study; as such, drawing conclusions is difficult. Furthermore, the low 30-day all-cause mortality rate of 2.6% in the positive troponin group may have been affected by close monitoring of the patients, who commonly received hemodynamic and oxygen support. Based on these factors, our conclusion is relatively weak, and we cannot recommend a change in practice compared to existing guidelines. In general, additional prospective research is needed to determine whether patients with low-risk PE tested positive for troponin can receive care safely outside the hospital or, rather, require hospitalization similar to patients with intermediate-high risk PE.

We identified a number of other limitations in our analysis. First, aside from the relatively small number of pertinent studies in the literature, most of the studies are of low-moderate quality. Second, the troponin classification in various studies was not conducted using the same assay, and the cut-off value determining positive versus negative results in each case may have differed. These differences may have created some ambiguity or misclassification when the data were pooled together. Third, although the mixed effects logistic regression model controls for some of the variations among patients enrolled in different studies, significant differences exist in terms of patient characteristics or the protocol for follow-up care. This aspect was unaccounted for in this analysis. Lastly, pooled outcome events could not be retrieved from all of the included studies, which would have resulted in a misrepresentation of the true outcomes.

The ESC guidelines suggest avoiding cardiac biomarker testing in patients with low-risk PE because this practice does not have therapeutic implications. Moreover, ESC and ACCP guidelines both state that a positive cardiac biomarker should discourage treatment out of the hospital. The ACCP guidelines further encourage testing of cardiac biomarkers and/or evaluating right ventricular function via echocardiography when uncertainty exists regarding whether patients may require close in-hospital monitoring or not. Although no resounding evidence suggests that troponins have therapeutic implications in patients with low-risk PE, the current guidelines and our meta-analysis cannot offer an overwhelmingly convincing recommendation about whether or not patients with low-risk PE and positive cardiac biomarkers are best treated in the ambulatory or inpatient setting. Such patients may benefit from monitoring in an observation unit (eg, less than 24 or 48 hours), rather than requiring a full admission to the hospital. Nevertheless, our analysis shows that making this determination will require prospective studies that will utilize cardiac troponin status in predicting PE-related events, such as arrhythmia, acute respiratory failure, and hemodynamic decompensation, rather than all-cause mortality.

Until further studies, hospitalists should integrate the use of cardiac troponin and other clinical data, including those available from patient history, physical exam, and other laboratory testing, in determining whether or not to admit, observe, or discharge patients with low-risk PE. As the current guidelines recommend, we support consideration of right ventricular function assessment, via echocardiogram or computed tomography, in patients with positive cardiac troponins even when their PESI/sPESI score is low.

The authors would like to thank Megan Therese Smith, PhD and Lishi Zhang, MS for their contribution in providing a comprehensive statistical analysis of this meta-analysis.

Disclosures

The authors declare no conflicts of interest in the work under consideration for publication. Abdullah Mahayni and Mukti Patel, MD also declared no conflicts of interest with regard to the relevant financial activities outside the submitted work. Omar Darwish, DO and Alpesh Amin, MD also declared no relevant financial activities outside the submitted work; they are speakers for Bristol Myer Squibb and Pfizer regarding the anticoagulant, Apixaban, for treatment of venous thromboembolism and atrial fibrillation.

Approximately 14% of children who sustain a concussion are admitted to the hospital,¹ although admission rates reportedly vary substantially among pediatric hospitals.² Children hospitalized for concussion may be at a higher risk for persistent postconcussive symptoms,^3,4 yet little is known about this subset of children and how they are managed while in the hospital. Characterizing children hospitalized for concussion and describing the inpatient care they received will promote hypothesis generation for further inquiry into indications for admission, as well as the relationship between inpatient management and concussion recovery.

We described a cohort of children admitted to 40 pediatric hospitals primarily for concussion and detailed care delivered during hospitalization. We explored individual-level factors and their association with prolonged length of stay (LOS) and emergency department (ED) readmission. Finally, we evaluated if there had been changes in inpatient care over the 8-year study period.

Study Design

The Institutional Review Board determined that this retrospective cohort study was exempt from review.

Data Source

The Children’s Hospital Association’s Pediatric Health Information System (PHIS) is an administrative database from pediatric hospitals located within 17 major metropolitan areas in the United States. Data include: service dates, patient demographics, payer type, diagnosis codes, resource utilization information (eg, medications), and hospital characteristics.^1,5 De-identified data undergo reliability and validity checks prior to inclusion.^1,5 We analyzed data from 40 of 43 hospitals that contributed inpatient data during our study period. 2 hospitals were excluded due to inconsistent data submission, and 1 removed their data.

Study Population

Data were extracted for children 0 to 17 years old who were admitted to an inpatient or observational unit between January 1, 2007 and December 31, 2014 for traumatic brain injury (TBI). Children were identified using International Classification of Diseases, Clinical Modification, Ninth Revision (ICD-9-CM) diagnosis codes that denote TBI per the Centers for Disease Control (CDC): 800.0–801.9, 803.0–804.9, 850–854.1, and 959.01.^6–8 To examine inpatient care for concussion, we only retained children with a primary (ie, first) concussion-related diagnosis code (850.0–850.9⁹) for analyses. For patients with multiple visits during our study period, only the index admission was analyzed. We refined our cohort using 2 injury scores calculated from ICD-9-CM diagnosis codes using validated ICDMAP-90 injury coding software.^6,10–12 The Abbreviated Injury Scale (AIS) ranges from 1 (minor injury) to 6 (not survivable). The total Injury Severity Score (ISS) is based on 6 body regions (head/neck, face, chest, abdomen, extremity, and external) and calculated by summing the squares of the 3 worst AIS scores.¹³ A concussion receives a head AIS score of 2 if there is an associated loss of consciousness or a score of 1 if there is not; therefore, children were excluded if the head AIS score was >2. We also excluded children with the following features, as they may be indicative of more severe injuries that were likely the cause of admission: ISS > 6, secondary diagnosis code of skull fracture or intracranial injury, intensive care unit (ICU) or operating room (OR) charges, or a LOS > 7 days. Because some children are hospitalized for potentially abusive minor head trauma pending a safe discharge plan, we excluded children 0 to 4 years of age with child abuse, which was determined using a specific set of diagnosis codes (E960-E96820, 995.54, and 995.55) similar to previous research.¹⁴

Data Elements and Outcomes

Outcomes

Based on previous reports,^1,15 a LOS ≥ 2 days distinguished a typical hospitalization from a prolonged one. ED revisit was identified when a child had a visit with a TBI-related primary diagnosis code at a PHIS hospital within 30 days of initial admission and was discharged home. We limited analyses to children discharged, as children readmitted may have had an initially missed intracranial injury.

Patient Characteristics

We examined the following patient variables: age, race, sex, presence of chronic medical condition, payer type, household income, area of residence (eg, rural versus urban), and mechanism of injury. Age was categorized to represent early childhood (0 to 4 years), school age (5 to 12 years), and adolescence (12 to 17 years). Race was grouped as white, black, or other (Asian, Pacific Islander, American Indian, and “other” per PHIS). Ethnicity was described as Hispanic/Latino or not Hispanic/Latino. Children with medical conditions lasting at least 12 months and comorbidities that may impact TBI recovery were identified using a subgrouping of ICD-9-CM codes for children with “complex chronic conditions”.¹⁶ Payer type was categorized as government, private, and self-pay. We extracted a PHIS variable representing the 2010 median household income for the child’s home zip code and categorized it into quartiles based on the Federal Poverty Level for a family of 4.^17,18 Area of residence was defined using a Rural–Urban Commuting Area (RUCA) classification system¹⁹ and grouped into large urban core, suburban area, large rural town, or small rural town/isolated rural area.¹⁷ Mechanism of injury was determined using E-codes and categorized using the CDC injury framework,²⁰ with sports-related injuries identified using a previously described set of E-codes.¹ Mechanisms of injury included fall, motor vehicle collision, other motorized transport (eg, all-terrain vehicles), sports-related, struck by or against (ie, objects), and all others (eg, cyclists).

Hospital Characteristics

Hospitals were characterized by region (Northeast, Central, South, and West) and size (small <200, medium 200–400, and large >400 beds). The trauma-level accreditation was identified with Level 1 reflecting the highest possible trauma resources.

Medical Care Variables

Care variables included medications, neuroimaging, and cost of stay. Medication classes included oral non-narcotic analgesics [acetaminophen, ibuprofen, and others (aspirin, tramadol, and naproxen)], oral narcotics (codeine, oxycodone, and narcotic–non-narcotic combinations), intravenous (IV) non-narcotics (ketorolac), IV narcotics (morphine, fentanyl, and hydromorphone), antiemetics [ondansetron, metoclopramide, and phenothiazines (prochlorperazine, chlorpromazine, and promethazine)], maintenance IV fluids (dextrose with electrolytes or 0.45% sodium chloride), and resuscitation IV fluids (0.9% sodium chloride or lactated Ringer’s solution). Receipt of neuroimaging was determined if head computed tomography (CT) had been conducted at the admitting hospital. Adjusted cost of stay was calculated using a hospital-specific cost-to-charge ratio with additional adjustments using the Center for Medicare & Medicaid’s Wage Index.

Statistical Analyses

Descriptive statistics were calculated for individual, injury, and hospital, and care data elements, LOS, and ED readmissions. The number of children admitted with TBI was used as the denominator to assess the proportion of pediatric TBI admissions that were due to concussions. To identify factors associated with prolonged LOS (ie, ≥2 days) and ED readmission, we employed a mixed models approach that accounted for clustering of observations within hospitals. Independent variables included age, sex, race, ethnicity, payer type, household income, RUCA code, chronic medical condition, and injury mechanism. Models were adjusted for hospital location, size, and trauma-level accreditation. The binary distribution was specified along with a logit link function. A 2-phase process determined factors associated with each outcome. First, bivariable models were developed, followed by multivariable models that included independent variables with P values < .25 in the bivariable analysis. Backward step-wise elimination was performed, deleting variables with the highest P value one at a time. After each deletion, the percentage change in odds ratios was examined; if variable removal resulted in >10% change, the variable was retained as a potential confounder. This process was repeated until all remaining variables were significant (P < .05) with the exception of potential confounders. Finally, we examined the proportion of children receiving selected care practices annually. Descriptive and trend analyses were used to analyze adjusted median cost of stay. Analyses were performed using SAS software (Version 9.3, SAS Institute Inc., Cary, North Carolina).

Over 8 years, 88,526 children were admitted to 40 PHIS hospitals with a TBI-related diagnosis, among whom 13,708 had a primary diagnosis of concussion. We excluded 2,973 children with 1 or more of the following characteristics: a secondary diagnosis of intracranial injury (n = 58), head AIS score > 2 (n = 218), LOS > 7 days (n = 50), OR charges (n = 132), ICU charges (n = 1947), and ISS > 6 (n = 568). Six additional children aging 0 to 4 years were excluded due to child abuse. The remaining 10,729 children, averaging 1300 hospitalizations annually, were identified as being hospitalized primarily for concussion.

Table 1 summarizes the individual characteristics for this cohort. The average (standard deviation) age was 9.5 (5.1) years. Ethnicity was missing for 25.3% and therefore excluded from the multivariable models. Almost all children had a head AIS score of 2 (99.2%), and the majority had a total ISS ≤ 4 (73.4%). The majority of admissions were admitted to Level 1 trauma-accredited hospitals (78.7%) and medium-sized hospitals (63.9%).

From 2007 to 2014, approximately 15% of children admitted to PHIS hospitals for TBI were admitted primarily for concussion. Since almost all children had a head AIS score of 2 and an ISS ≤ 4, our data suggest that most children had an associated loss of consciousness and that concussion was the only injury sustained, respectively. This study identified important subgroups that necessitated inpatient care but are rarely the focus of concussion research (eg, toddlers and those injured due to a motor vehicle collision). Most children (83.3%) received medications to treat common postconcussive symptoms (eg, pain and nausea), with almost half receiving 3 or more medication classes. Factors associated with the development of postconcussive syndrome (eg, female sex and adolescent age)^4,21 were significantly associated with hospitalization of 2 or more days and ED revisit within 30 days of admission. In the absence of evidenced-based guidelines for inpatient concussion management, we identified significant trends in care, including increased use of specific pain [ie, oral and IV nonsteroidal anti-inflammatory drugs (NSAIDs)] and antiemetic (ie, ondansetron) medications and decreased use of head CT. Given the number of children admitted and receiving treatment for concussion symptomatology, influences on the decision to deliver specific care practices, as well as the impact and benefit of hospitalization, require closer examination.

Our study extends previous reports from the PHIS database by characterizing children admitted for concussion.¹ We found that children admitted for concussion had similar characteristics to the broader population of children who sustain concussion (eg, school-aged children, male, and injured due to a fall or during sports).^1,3,22 However, approximately 20% of the cohort were less than 5 years old, and less is known regarding appropriate treatment and outcomes of concussion in this age group.²³ Uncertainty regarding optimal management and a young child’s inability to articulate symptoms may contribute to a physician’s decision to admit for close observation. Similar to Blinman et al., we found that a substantial proportion of children admitted with concussion were injured due to a motor vehicle collision,³ suggesting that although sports-related injuries are responsible for a significant proportion of pediatric concussions, children injured by other preventable mechanisms may also be incurring significant concussive injuries. Finally, the majority of our cohort was from an urban core, relative to a rural area, which is likely a reflection of the regionalization of trauma care, as well as variations in access to health care.

Although most children recover fully from concussion without specific interventions, 20%-30% may remain symptomatic at 1 month,^3,4,21,24 and children who are hospitalized with concussion may be at higher risk for protracted symptoms. While specific individual or injury-related factors (eg, female sex, adolescent age, and injury due to motor vehicle collision) may contribute to more significant postconcussive symptoms, it is unclear how inpatient management affects recovery trajectory. Frequent sleep disruptions associated with inpatient care²⁵ contradict current acute concussion management recommendations for physical and cognitive rest²⁶ and could potentially impair symptom recovery. Additionally, we found widespread use of NSAIDs, although there is evidence suggesting that NSAIDs may potentially worsen concussive symptoms.²⁶ We identified an increase in medication usage over time despite limited evidence of their effectiveness for pediatric concussion.^27–29 This change may reflect improved symptom screening^4,30 and/or increased awareness of specific medication safety profiles in pediatric trauma patients, especially for NSAIDs and ondansetron. Although we saw an increase in NSAID use, we did not see a proportional decrease in narcotic use. Similarly, while two-thirds of our cohort received IV medications, there is controversy about the need for IV fluids and medications for other pediatric illnesses, with research demonstrating that IV treatment may not reduce recovery time and may contribute to prolonged hospitalization and phlebitis.^31,32 Thus, there is a need to understand the therapeutic effectiveness and benefits of medications and fluids on postconcussion recovery.

Neuroimaging rates for children receiving ED evaluation for concussion have been reported to be up to 60%-70%,^1,22 although a more recent study spanning 2006 to 2011 found a 35-%–40% head CT rate in pediatric patients by hospital-based EDs in the United States.³³ Our results appear to support decreasing head CT use over time in pediatric hospitals. Hospitalization for observation is costly¹ but could decrease a child’s risk of malignancy from radiation exposure. Further work on balancing cost, risk, and shared decision-making with parents could guide decisions regarding emergent neuroimaging versus admission.

This study has limitations inherent to the use of an administrative dataset, including lack of information regarding why the child was admitted. Since the focus was to describe inpatient care of children with concussion, those discharged home from the ED were not included in this dataset. Consequently, we could not contrast the ED care of those discharged home with those who were admitted or assess trends in admission rates for concussion. Although the overall number of concussion admissions has continued to remain stable over time,¹ due to a lack of prospectively collected clinical information, we are unable to determine whether observed trends in care are secondary to changes in practice or changes in concussion severity. However, there has been no research to date supporting the latter. Ethnicity was excluded due to high levels of missing data. Cost of stay was not extensively analyzed given hospital variation in designation of observational or inpatient status, which subsequently affects billing.³⁴ Rates of neuroimaging and ED revisit may have been underestimated since children could have received care at a non-PHIS hospital. Similarly, the decrease in the proportion of children receiving neuroimaging over time may have been associated with an increase in children being transferred from a non-PHIS hospital for admission, although with increased regionalization in trauma care, we would not expect transfers of children with only concussion to have significantly increased. Finally, data were limited to the pediatric tertiary care centers participating in PHIS, thereby reducing generalizability and introducing selection bias by only including children who were able to access care at PHIS hospitals. Although the care practices we evaluated (eg, NSAIDs and head CT) are available at all hospitals, our analyses only reflect care delivered within the PHIS.

Concussion accounted for 15% of all pediatric TBI admissions during our study period. Further investigation of potential factors associated with admission and protracted recovery (eg, adolescent females needing treatment for severe symptomatology) could facilitate better understanding of how hospitalization affects recovery. Additionally, research on acute pharmacotherapies (eg, IV therapies and/or inpatient treatment until symptoms resolve) is needed to fully elucidate the acute and long-term benefits of interventions delivered to children.

Colleen Mangeot: Biostatistician with extensive PHIS knowledge who contributed to database creation and statistical analysis. Yanhong (Amy) Liu: Research database programmer who developed the database, ran quality assurance measures, and cleaned all study data.

Disclosures

The authors have nothing to disclose.

Funding

This study was supported by grant R40 MC 268060102 from the Maternal and Child Health Research Program, Maternal and Child Health Bureau (Title V, Social Security Act), Health Resources and Services Administration, Department of Health and Human Services. The funding source was not involved in development of the study design; in the collection, analysis and interpretation of data; or in the writing of this report.

Approximately 14% of children who sustain a concussion are admitted to the hospital,¹ although admission rates reportedly vary substantially among pediatric hospitals.² Children hospitalized for concussion may be at a higher risk for persistent postconcussive symptoms,^3,4 yet little is known about this subset of children and how they are managed while in the hospital. Characterizing children hospitalized for concussion and describing the inpatient care they received will promote hypothesis generation for further inquiry into indications for admission, as well as the relationship between inpatient management and concussion recovery.

We described a cohort of children admitted to 40 pediatric hospitals primarily for concussion and detailed care delivered during hospitalization. We explored individual-level factors and their association with prolonged length of stay (LOS) and emergency department (ED) readmission. Finally, we evaluated if there had been changes in inpatient care over the 8-year study period.

Study Design

The Institutional Review Board determined that this retrospective cohort study was exempt from review.

Data Source

The Children’s Hospital Association’s Pediatric Health Information System (PHIS) is an administrative database from pediatric hospitals located within 17 major metropolitan areas in the United States. Data include: service dates, patient demographics, payer type, diagnosis codes, resource utilization information (eg, medications), and hospital characteristics.^1,5 De-identified data undergo reliability and validity checks prior to inclusion.^1,5 We analyzed data from 40 of 43 hospitals that contributed inpatient data during our study period. 2 hospitals were excluded due to inconsistent data submission, and 1 removed their data.

Study Population

Data were extracted for children 0 to 17 years old who were admitted to an inpatient or observational unit between January 1, 2007 and December 31, 2014 for traumatic brain injury (TBI). Children were identified using International Classification of Diseases, Clinical Modification, Ninth Revision (ICD-9-CM) diagnosis codes that denote TBI per the Centers for Disease Control (CDC): 800.0–801.9, 803.0–804.9, 850–854.1, and 959.01.^6–8 To examine inpatient care for concussion, we only retained children with a primary (ie, first) concussion-related diagnosis code (850.0–850.9⁹) for analyses. For patients with multiple visits during our study period, only the index admission was analyzed. We refined our cohort using 2 injury scores calculated from ICD-9-CM diagnosis codes using validated ICDMAP-90 injury coding software.^6,10–12 The Abbreviated Injury Scale (AIS) ranges from 1 (minor injury) to 6 (not survivable). The total Injury Severity Score (ISS) is based on 6 body regions (head/neck, face, chest, abdomen, extremity, and external) and calculated by summing the squares of the 3 worst AIS scores.¹³ A concussion receives a head AIS score of 2 if there is an associated loss of consciousness or a score of 1 if there is not; therefore, children were excluded if the head AIS score was >2. We also excluded children with the following features, as they may be indicative of more severe injuries that were likely the cause of admission: ISS > 6, secondary diagnosis code of skull fracture or intracranial injury, intensive care unit (ICU) or operating room (OR) charges, or a LOS > 7 days. Because some children are hospitalized for potentially abusive minor head trauma pending a safe discharge plan, we excluded children 0 to 4 years of age with child abuse, which was determined using a specific set of diagnosis codes (E960-E96820, 995.54, and 995.55) similar to previous research.¹⁴

Data Elements and Outcomes

Outcomes

Based on previous reports,^1,15 a LOS ≥ 2 days distinguished a typical hospitalization from a prolonged one. ED revisit was identified when a child had a visit with a TBI-related primary diagnosis code at a PHIS hospital within 30 days of initial admission and was discharged home. We limited analyses to children discharged, as children readmitted may have had an initially missed intracranial injury.

Patient Characteristics

We examined the following patient variables: age, race, sex, presence of chronic medical condition, payer type, household income, area of residence (eg, rural versus urban), and mechanism of injury. Age was categorized to represent early childhood (0 to 4 years), school age (5 to 12 years), and adolescence (12 to 17 years). Race was grouped as white, black, or other (Asian, Pacific Islander, American Indian, and “other” per PHIS). Ethnicity was described as Hispanic/Latino or not Hispanic/Latino. Children with medical conditions lasting at least 12 months and comorbidities that may impact TBI recovery were identified using a subgrouping of ICD-9-CM codes for children with “complex chronic conditions”.¹⁶ Payer type was categorized as government, private, and self-pay. We extracted a PHIS variable representing the 2010 median household income for the child’s home zip code and categorized it into quartiles based on the Federal Poverty Level for a family of 4.^17,18 Area of residence was defined using a Rural–Urban Commuting Area (RUCA) classification system¹⁹ and grouped into large urban core, suburban area, large rural town, or small rural town/isolated rural area.¹⁷ Mechanism of injury was determined using E-codes and categorized using the CDC injury framework,²⁰ with sports-related injuries identified using a previously described set of E-codes.¹ Mechanisms of injury included fall, motor vehicle collision, other motorized transport (eg, all-terrain vehicles), sports-related, struck by or against (ie, objects), and all others (eg, cyclists).

Hospital Characteristics

Hospitals were characterized by region (Northeast, Central, South, and West) and size (small <200, medium 200–400, and large >400 beds). The trauma-level accreditation was identified with Level 1 reflecting the highest possible trauma resources.

Medical Care Variables

Care variables included medications, neuroimaging, and cost of stay. Medication classes included oral non-narcotic analgesics [acetaminophen, ibuprofen, and others (aspirin, tramadol, and naproxen)], oral narcotics (codeine, oxycodone, and narcotic–non-narcotic combinations), intravenous (IV) non-narcotics (ketorolac), IV narcotics (morphine, fentanyl, and hydromorphone), antiemetics [ondansetron, metoclopramide, and phenothiazines (prochlorperazine, chlorpromazine, and promethazine)], maintenance IV fluids (dextrose with electrolytes or 0.45% sodium chloride), and resuscitation IV fluids (0.9% sodium chloride or lactated Ringer’s solution). Receipt of neuroimaging was determined if head computed tomography (CT) had been conducted at the admitting hospital. Adjusted cost of stay was calculated using a hospital-specific cost-to-charge ratio with additional adjustments using the Center for Medicare & Medicaid’s Wage Index.

Statistical Analyses

Descriptive statistics were calculated for individual, injury, and hospital, and care data elements, LOS, and ED readmissions. The number of children admitted with TBI was used as the denominator to assess the proportion of pediatric TBI admissions that were due to concussions. To identify factors associated with prolonged LOS (ie, ≥2 days) and ED readmission, we employed a mixed models approach that accounted for clustering of observations within hospitals. Independent variables included age, sex, race, ethnicity, payer type, household income, RUCA code, chronic medical condition, and injury mechanism. Models were adjusted for hospital location, size, and trauma-level accreditation. The binary distribution was specified along with a logit link function. A 2-phase process determined factors associated with each outcome. First, bivariable models were developed, followed by multivariable models that included independent variables with P values < .25 in the bivariable analysis. Backward step-wise elimination was performed, deleting variables with the highest P value one at a time. After each deletion, the percentage change in odds ratios was examined; if variable removal resulted in >10% change, the variable was retained as a potential confounder. This process was repeated until all remaining variables were significant (P < .05) with the exception of potential confounders. Finally, we examined the proportion of children receiving selected care practices annually. Descriptive and trend analyses were used to analyze adjusted median cost of stay. Analyses were performed using SAS software (Version 9.3, SAS Institute Inc., Cary, North Carolina).

Over 8 years, 88,526 children were admitted to 40 PHIS hospitals with a TBI-related diagnosis, among whom 13,708 had a primary diagnosis of concussion. We excluded 2,973 children with 1 or more of the following characteristics: a secondary diagnosis of intracranial injury (n = 58), head AIS score > 2 (n = 218), LOS > 7 days (n = 50), OR charges (n = 132), ICU charges (n = 1947), and ISS > 6 (n = 568). Six additional children aging 0 to 4 years were excluded due to child abuse. The remaining 10,729 children, averaging 1300 hospitalizations annually, were identified as being hospitalized primarily for concussion.

Table 1 summarizes the individual characteristics for this cohort. The average (standard deviation) age was 9.5 (5.1) years. Ethnicity was missing for 25.3% and therefore excluded from the multivariable models. Almost all children had a head AIS score of 2 (99.2%), and the majority had a total ISS ≤ 4 (73.4%). The majority of admissions were admitted to Level 1 trauma-accredited hospitals (78.7%) and medium-sized hospitals (63.9%).

From 2007 to 2014, approximately 15% of children admitted to PHIS hospitals for TBI were admitted primarily for concussion. Since almost all children had a head AIS score of 2 and an ISS ≤ 4, our data suggest that most children had an associated loss of consciousness and that concussion was the only injury sustained, respectively. This study identified important subgroups that necessitated inpatient care but are rarely the focus of concussion research (eg, toddlers and those injured due to a motor vehicle collision). Most children (83.3%) received medications to treat common postconcussive symptoms (eg, pain and nausea), with almost half receiving 3 or more medication classes. Factors associated with the development of postconcussive syndrome (eg, female sex and adolescent age)^4,21 were significantly associated with hospitalization of 2 or more days and ED revisit within 30 days of admission. In the absence of evidenced-based guidelines for inpatient concussion management, we identified significant trends in care, including increased use of specific pain [ie, oral and IV nonsteroidal anti-inflammatory drugs (NSAIDs)] and antiemetic (ie, ondansetron) medications and decreased use of head CT. Given the number of children admitted and receiving treatment for concussion symptomatology, influences on the decision to deliver specific care practices, as well as the impact and benefit of hospitalization, require closer examination.

Our study extends previous reports from the PHIS database by characterizing children admitted for concussion.¹ We found that children admitted for concussion had similar characteristics to the broader population of children who sustain concussion (eg, school-aged children, male, and injured due to a fall or during sports).^1,3,22 However, approximately 20% of the cohort were less than 5 years old, and less is known regarding appropriate treatment and outcomes of concussion in this age group.²³ Uncertainty regarding optimal management and a young child’s inability to articulate symptoms may contribute to a physician’s decision to admit for close observation. Similar to Blinman et al., we found that a substantial proportion of children admitted with concussion were injured due to a motor vehicle collision,³ suggesting that although sports-related injuries are responsible for a significant proportion of pediatric concussions, children injured by other preventable mechanisms may also be incurring significant concussive injuries. Finally, the majority of our cohort was from an urban core, relative to a rural area, which is likely a reflection of the regionalization of trauma care, as well as variations in access to health care.

Although most children recover fully from concussion without specific interventions, 20%-30% may remain symptomatic at 1 month,^3,4,21,24 and children who are hospitalized with concussion may be at higher risk for protracted symptoms. While specific individual or injury-related factors (eg, female sex, adolescent age, and injury due to motor vehicle collision) may contribute to more significant postconcussive symptoms, it is unclear how inpatient management affects recovery trajectory. Frequent sleep disruptions associated with inpatient care²⁵ contradict current acute concussion management recommendations for physical and cognitive rest²⁶ and could potentially impair symptom recovery. Additionally, we found widespread use of NSAIDs, although there is evidence suggesting that NSAIDs may potentially worsen concussive symptoms.²⁶ We identified an increase in medication usage over time despite limited evidence of their effectiveness for pediatric concussion.^27–29 This change may reflect improved symptom screening^4,30 and/or increased awareness of specific medication safety profiles in pediatric trauma patients, especially for NSAIDs and ondansetron. Although we saw an increase in NSAID use, we did not see a proportional decrease in narcotic use. Similarly, while two-thirds of our cohort received IV medications, there is controversy about the need for IV fluids and medications for other pediatric illnesses, with research demonstrating that IV treatment may not reduce recovery time and may contribute to prolonged hospitalization and phlebitis.^31,32 Thus, there is a need to understand the therapeutic effectiveness and benefits of medications and fluids on postconcussion recovery.

Neuroimaging rates for children receiving ED evaluation for concussion have been reported to be up to 60%-70%,^1,22 although a more recent study spanning 2006 to 2011 found a 35-%–40% head CT rate in pediatric patients by hospital-based EDs in the United States.³³ Our results appear to support decreasing head CT use over time in pediatric hospitals. Hospitalization for observation is costly¹ but could decrease a child’s risk of malignancy from radiation exposure. Further work on balancing cost, risk, and shared decision-making with parents could guide decisions regarding emergent neuroimaging versus admission.

This study has limitations inherent to the use of an administrative dataset, including lack of information regarding why the child was admitted. Since the focus was to describe inpatient care of children with concussion, those discharged home from the ED were not included in this dataset. Consequently, we could not contrast the ED care of those discharged home with those who were admitted or assess trends in admission rates for concussion. Although the overall number of concussion admissions has continued to remain stable over time,¹ due to a lack of prospectively collected clinical information, we are unable to determine whether observed trends in care are secondary to changes in practice or changes in concussion severity. However, there has been no research to date supporting the latter. Ethnicity was excluded due to high levels of missing data. Cost of stay was not extensively analyzed given hospital variation in designation of observational or inpatient status, which subsequently affects billing.³⁴ Rates of neuroimaging and ED revisit may have been underestimated since children could have received care at a non-PHIS hospital. Similarly, the decrease in the proportion of children receiving neuroimaging over time may have been associated with an increase in children being transferred from a non-PHIS hospital for admission, although with increased regionalization in trauma care, we would not expect transfers of children with only concussion to have significantly increased. Finally, data were limited to the pediatric tertiary care centers participating in PHIS, thereby reducing generalizability and introducing selection bias by only including children who were able to access care at PHIS hospitals. Although the care practices we evaluated (eg, NSAIDs and head CT) are available at all hospitals, our analyses only reflect care delivered within the PHIS.

Concussion accounted for 15% of all pediatric TBI admissions during our study period. Further investigation of potential factors associated with admission and protracted recovery (eg, adolescent females needing treatment for severe symptomatology) could facilitate better understanding of how hospitalization affects recovery. Additionally, research on acute pharmacotherapies (eg, IV therapies and/or inpatient treatment until symptoms resolve) is needed to fully elucidate the acute and long-term benefits of interventions delivered to children.

Colleen Mangeot: Biostatistician with extensive PHIS knowledge who contributed to database creation and statistical analysis. Yanhong (Amy) Liu: Research database programmer who developed the database, ran quality assurance measures, and cleaned all study data.

Disclosures

The authors have nothing to disclose.

Funding

This study was supported by grant R40 MC 268060102 from the Maternal and Child Health Research Program, Maternal and Child Health Bureau (Title V, Social Security Act), Health Resources and Services Administration, Department of Health and Human Services. The funding source was not involved in development of the study design; in the collection, analysis and interpretation of data; or in the writing of this report.

Approximately 14% of children who sustain a concussion are admitted to the hospital,¹ although admission rates reportedly vary substantially among pediatric hospitals.² Children hospitalized for concussion may be at a higher risk for persistent postconcussive symptoms,^3,4 yet little is known about this subset of children and how they are managed while in the hospital. Characterizing children hospitalized for concussion and describing the inpatient care they received will promote hypothesis generation for further inquiry into indications for admission, as well as the relationship between inpatient management and concussion recovery.

We described a cohort of children admitted to 40 pediatric hospitals primarily for concussion and detailed care delivered during hospitalization. We explored individual-level factors and their association with prolonged length of stay (LOS) and emergency department (ED) readmission. Finally, we evaluated if there had been changes in inpatient care over the 8-year study period.

Study Design

The Institutional Review Board determined that this retrospective cohort study was exempt from review.

Data Source

The Children’s Hospital Association’s Pediatric Health Information System (PHIS) is an administrative database from pediatric hospitals located within 17 major metropolitan areas in the United States. Data include: service dates, patient demographics, payer type, diagnosis codes, resource utilization information (eg, medications), and hospital characteristics.^1,5 De-identified data undergo reliability and validity checks prior to inclusion.^1,5 We analyzed data from 40 of 43 hospitals that contributed inpatient data during our study period. 2 hospitals were excluded due to inconsistent data submission, and 1 removed their data.

Study Population

Data were extracted for children 0 to 17 years old who were admitted to an inpatient or observational unit between January 1, 2007 and December 31, 2014 for traumatic brain injury (TBI). Children were identified using International Classification of Diseases, Clinical Modification, Ninth Revision (ICD-9-CM) diagnosis codes that denote TBI per the Centers for Disease Control (CDC): 800.0–801.9, 803.0–804.9, 850–854.1, and 959.01.^6–8 To examine inpatient care for concussion, we only retained children with a primary (ie, first) concussion-related diagnosis code (850.0–850.9⁹) for analyses. For patients with multiple visits during our study period, only the index admission was analyzed. We refined our cohort using 2 injury scores calculated from ICD-9-CM diagnosis codes using validated ICDMAP-90 injury coding software.^6,10–12 The Abbreviated Injury Scale (AIS) ranges from 1 (minor injury) to 6 (not survivable). The total Injury Severity Score (ISS) is based on 6 body regions (head/neck, face, chest, abdomen, extremity, and external) and calculated by summing the squares of the 3 worst AIS scores.¹³ A concussion receives a head AIS score of 2 if there is an associated loss of consciousness or a score of 1 if there is not; therefore, children were excluded if the head AIS score was >2. We also excluded children with the following features, as they may be indicative of more severe injuries that were likely the cause of admission: ISS > 6, secondary diagnosis code of skull fracture or intracranial injury, intensive care unit (ICU) or operating room (OR) charges, or a LOS > 7 days. Because some children are hospitalized for potentially abusive minor head trauma pending a safe discharge plan, we excluded children 0 to 4 years of age with child abuse, which was determined using a specific set of diagnosis codes (E960-E96820, 995.54, and 995.55) similar to previous research.¹⁴

Data Elements and Outcomes

Outcomes

Based on previous reports,^1,15 a LOS ≥ 2 days distinguished a typical hospitalization from a prolonged one. ED revisit was identified when a child had a visit with a TBI-related primary diagnosis code at a PHIS hospital within 30 days of initial admission and was discharged home. We limited analyses to children discharged, as children readmitted may have had an initially missed intracranial injury.

Patient Characteristics

We examined the following patient variables: age, race, sex, presence of chronic medical condition, payer type, household income, area of residence (eg, rural versus urban), and mechanism of injury. Age was categorized to represent early childhood (0 to 4 years), school age (5 to 12 years), and adolescence (12 to 17 years). Race was grouped as white, black, or other (Asian, Pacific Islander, American Indian, and “other” per PHIS). Ethnicity was described as Hispanic/Latino or not Hispanic/Latino. Children with medical conditions lasting at least 12 months and comorbidities that may impact TBI recovery were identified using a subgrouping of ICD-9-CM codes for children with “complex chronic conditions”.¹⁶ Payer type was categorized as government, private, and self-pay. We extracted a PHIS variable representing the 2010 median household income for the child’s home zip code and categorized it into quartiles based on the Federal Poverty Level for a family of 4.^17,18 Area of residence was defined using a Rural–Urban Commuting Area (RUCA) classification system¹⁹ and grouped into large urban core, suburban area, large rural town, or small rural town/isolated rural area.¹⁷ Mechanism of injury was determined using E-codes and categorized using the CDC injury framework,²⁰ with sports-related injuries identified using a previously described set of E-codes.¹ Mechanisms of injury included fall, motor vehicle collision, other motorized transport (eg, all-terrain vehicles), sports-related, struck by or against (ie, objects), and all others (eg, cyclists).

Hospital Characteristics

Hospitals were characterized by region (Northeast, Central, South, and West) and size (small <200, medium 200–400, and large >400 beds). The trauma-level accreditation was identified with Level 1 reflecting the highest possible trauma resources.

Medical Care Variables

Care variables included medications, neuroimaging, and cost of stay. Medication classes included oral non-narcotic analgesics [acetaminophen, ibuprofen, and others (aspirin, tramadol, and naproxen)], oral narcotics (codeine, oxycodone, and narcotic–non-narcotic combinations), intravenous (IV) non-narcotics (ketorolac), IV narcotics (morphine, fentanyl, and hydromorphone), antiemetics [ondansetron, metoclopramide, and phenothiazines (prochlorperazine, chlorpromazine, and promethazine)], maintenance IV fluids (dextrose with electrolytes or 0.45% sodium chloride), and resuscitation IV fluids (0.9% sodium chloride or lactated Ringer’s solution). Receipt of neuroimaging was determined if head computed tomography (CT) had been conducted at the admitting hospital. Adjusted cost of stay was calculated using a hospital-specific cost-to-charge ratio with additional adjustments using the Center for Medicare & Medicaid’s Wage Index.

Statistical Analyses

Descriptive statistics were calculated for individual, injury, and hospital, and care data elements, LOS, and ED readmissions. The number of children admitted with TBI was used as the denominator to assess the proportion of pediatric TBI admissions that were due to concussions. To identify factors associated with prolonged LOS (ie, ≥2 days) and ED readmission, we employed a mixed models approach that accounted for clustering of observations within hospitals. Independent variables included age, sex, race, ethnicity, payer type, household income, RUCA code, chronic medical condition, and injury mechanism. Models were adjusted for hospital location, size, and trauma-level accreditation. The binary distribution was specified along with a logit link function. A 2-phase process determined factors associated with each outcome. First, bivariable models were developed, followed by multivariable models that included independent variables with P values < .25 in the bivariable analysis. Backward step-wise elimination was performed, deleting variables with the highest P value one at a time. After each deletion, the percentage change in odds ratios was examined; if variable removal resulted in >10% change, the variable was retained as a potential confounder. This process was repeated until all remaining variables were significant (P < .05) with the exception of potential confounders. Finally, we examined the proportion of children receiving selected care practices annually. Descriptive and trend analyses were used to analyze adjusted median cost of stay. Analyses were performed using SAS software (Version 9.3, SAS Institute Inc., Cary, North Carolina).

Over 8 years, 88,526 children were admitted to 40 PHIS hospitals with a TBI-related diagnosis, among whom 13,708 had a primary diagnosis of concussion. We excluded 2,973 children with 1 or more of the following characteristics: a secondary diagnosis of intracranial injury (n = 58), head AIS score > 2 (n = 218), LOS > 7 days (n = 50), OR charges (n = 132), ICU charges (n = 1947), and ISS > 6 (n = 568). Six additional children aging 0 to 4 years were excluded due to child abuse. The remaining 10,729 children, averaging 1300 hospitalizations annually, were identified as being hospitalized primarily for concussion.

Table 1 summarizes the individual characteristics for this cohort. The average (standard deviation) age was 9.5 (5.1) years. Ethnicity was missing for 25.3% and therefore excluded from the multivariable models. Almost all children had a head AIS score of 2 (99.2%), and the majority had a total ISS ≤ 4 (73.4%). The majority of admissions were admitted to Level 1 trauma-accredited hospitals (78.7%) and medium-sized hospitals (63.9%).

From 2007 to 2014, approximately 15% of children admitted to PHIS hospitals for TBI were admitted primarily for concussion. Since almost all children had a head AIS score of 2 and an ISS ≤ 4, our data suggest that most children had an associated loss of consciousness and that concussion was the only injury sustained, respectively. This study identified important subgroups that necessitated inpatient care but are rarely the focus of concussion research (eg, toddlers and those injured due to a motor vehicle collision). Most children (83.3%) received medications to treat common postconcussive symptoms (eg, pain and nausea), with almost half receiving 3 or more medication classes. Factors associated with the development of postconcussive syndrome (eg, female sex and adolescent age)^4,21 were significantly associated with hospitalization of 2 or more days and ED revisit within 30 days of admission. In the absence of evidenced-based guidelines for inpatient concussion management, we identified significant trends in care, including increased use of specific pain [ie, oral and IV nonsteroidal anti-inflammatory drugs (NSAIDs)] and antiemetic (ie, ondansetron) medications and decreased use of head CT. Given the number of children admitted and receiving treatment for concussion symptomatology, influences on the decision to deliver specific care practices, as well as the impact and benefit of hospitalization, require closer examination.

Our study extends previous reports from the PHIS database by characterizing children admitted for concussion.¹ We found that children admitted for concussion had similar characteristics to the broader population of children who sustain concussion (eg, school-aged children, male, and injured due to a fall or during sports).^1,3,22 However, approximately 20% of the cohort were less than 5 years old, and less is known regarding appropriate treatment and outcomes of concussion in this age group.²³ Uncertainty regarding optimal management and a young child’s inability to articulate symptoms may contribute to a physician’s decision to admit for close observation. Similar to Blinman et al., we found that a substantial proportion of children admitted with concussion were injured due to a motor vehicle collision,³ suggesting that although sports-related injuries are responsible for a significant proportion of pediatric concussions, children injured by other preventable mechanisms may also be incurring significant concussive injuries. Finally, the majority of our cohort was from an urban core, relative to a rural area, which is likely a reflection of the regionalization of trauma care, as well as variations in access to health care.

Although most children recover fully from concussion without specific interventions, 20%-30% may remain symptomatic at 1 month,^3,4,21,24 and children who are hospitalized with concussion may be at higher risk for protracted symptoms. While specific individual or injury-related factors (eg, female sex, adolescent age, and injury due to motor vehicle collision) may contribute to more significant postconcussive symptoms, it is unclear how inpatient management affects recovery trajectory. Frequent sleep disruptions associated with inpatient care²⁵ contradict current acute concussion management recommendations for physical and cognitive rest²⁶ and could potentially impair symptom recovery. Additionally, we found widespread use of NSAIDs, although there is evidence suggesting that NSAIDs may potentially worsen concussive symptoms.²⁶ We identified an increase in medication usage over time despite limited evidence of their effectiveness for pediatric concussion.^27–29 This change may reflect improved symptom screening^4,30 and/or increased awareness of specific medication safety profiles in pediatric trauma patients, especially for NSAIDs and ondansetron. Although we saw an increase in NSAID use, we did not see a proportional decrease in narcotic use. Similarly, while two-thirds of our cohort received IV medications, there is controversy about the need for IV fluids and medications for other pediatric illnesses, with research demonstrating that IV treatment may not reduce recovery time and may contribute to prolonged hospitalization and phlebitis.^31,32 Thus, there is a need to understand the therapeutic effectiveness and benefits of medications and fluids on postconcussion recovery.

Neuroimaging rates for children receiving ED evaluation for concussion have been reported to be up to 60%-70%,^1,22 although a more recent study spanning 2006 to 2011 found a 35-%–40% head CT rate in pediatric patients by hospital-based EDs in the United States.³³ Our results appear to support decreasing head CT use over time in pediatric hospitals. Hospitalization for observation is costly¹ but could decrease a child’s risk of malignancy from radiation exposure. Further work on balancing cost, risk, and shared decision-making with parents could guide decisions regarding emergent neuroimaging versus admission.

This study has limitations inherent to the use of an administrative dataset, including lack of information regarding why the child was admitted. Since the focus was to describe inpatient care of children with concussion, those discharged home from the ED were not included in this dataset. Consequently, we could not contrast the ED care of those discharged home with those who were admitted or assess trends in admission rates for concussion. Although the overall number of concussion admissions has continued to remain stable over time,¹ due to a lack of prospectively collected clinical information, we are unable to determine whether observed trends in care are secondary to changes in practice or changes in concussion severity. However, there has been no research to date supporting the latter. Ethnicity was excluded due to high levels of missing data. Cost of stay was not extensively analyzed given hospital variation in designation of observational or inpatient status, which subsequently affects billing.³⁴ Rates of neuroimaging and ED revisit may have been underestimated since children could have received care at a non-PHIS hospital. Similarly, the decrease in the proportion of children receiving neuroimaging over time may have been associated with an increase in children being transferred from a non-PHIS hospital for admission, although with increased regionalization in trauma care, we would not expect transfers of children with only concussion to have significantly increased. Finally, data were limited to the pediatric tertiary care centers participating in PHIS, thereby reducing generalizability and introducing selection bias by only including children who were able to access care at PHIS hospitals. Although the care practices we evaluated (eg, NSAIDs and head CT) are available at all hospitals, our analyses only reflect care delivered within the PHIS.

Concussion accounted for 15% of all pediatric TBI admissions during our study period. Further investigation of potential factors associated with admission and protracted recovery (eg, adolescent females needing treatment for severe symptomatology) could facilitate better understanding of how hospitalization affects recovery. Additionally, research on acute pharmacotherapies (eg, IV therapies and/or inpatient treatment until symptoms resolve) is needed to fully elucidate the acute and long-term benefits of interventions delivered to children.

Colleen Mangeot: Biostatistician with extensive PHIS knowledge who contributed to database creation and statistical analysis. Yanhong (Amy) Liu: Research database programmer who developed the database, ran quality assurance measures, and cleaned all study data.

Disclosures

The authors have nothing to disclose.

Funding

This study was supported by grant R40 MC 268060102 from the Maternal and Child Health Research Program, Maternal and Child Health Bureau (Title V, Social Security Act), Health Resources and Services Administration, Department of Health and Human Services. The funding source was not involved in development of the study design; in the collection, analysis and interpretation of data; or in the writing of this report.

Diagnostic error—defined as a failure to establish an accurate and timely explanation of the patient’s health problem—is an important source of patient harm.¹ Data suggest that all patients will experience at least 1 diagnostic error in their lifetime.^2-4 Not surprisingly, diagnostic errors are among the leading categories of paid malpractice claims in the United States.⁵

Despite diagnostic errors being morbid and sometimes deadly in the hospital,^6,7 little is known about how residents and learners approach diagnostic decision making. Errors in diagnosis are believed to stem from cognitive or system failures,⁸ with errors in cognition believed to occur due to rapid, reflexive thinking operating in the absence of a more analytical, deliberate process. System-based problems (eg, lack of expert availability, technology barriers, and access to data) have also been cited as contributors.⁹ However, whether and how these apply to trainees is not known.

Therefore, we conducted a focused ethnography of inpatient medicine teams (ie, attendings, residents, interns, and medical students) in 2 affiliated teaching hospitals, aiming to (a) observe the process of diagnosis by trainees and (b) identify methods to improve the diagnostic process and prevent errors.

We designed a multimethod, focused ethnographic study to examine diagnostic decision making in hospital settings.^10,11 In contrast to anthropologic ethnographies that study entire fields using open-ended questions, our study was designed to examine the process of diagnosis from the perspective of clinicians engaged in this activity.¹¹ This approach allowed us to capture diagnostic decisions and cognitive and system-based factors in a manner currently lacking in the literature.¹²

Setting and Participants

Between January 2016 and May 2016, we observed the members of four inpatient internal medicine teaching teams at 2 affiliated teaching hospitals. We purposefully selected teaching teams for observation because they are the primary model of care in academic settings and we have expertise in carrying out similar studies.^13,14 Teaching teams typically consisted of a medical attending (senior-level physician), 1 senior resident (a second- or third-year postgraduate trainee), two interns (a trainee in their first postgraduate year), and two to four  medical students. Teams were selected at random using existing schedules and followed Monday to Friday so as to permit observation of work on call and noncall days. Owing to manpower limitations, weekend and night shifts were not observed. However, overnight events were captured during morning rounds.

Most of the teams began rounds at 8:30 AM. Typically, rounds lasted for 90–120 min and concluded with a recap (ie, “running the list”) with a review of explicit plans for patients after they had been evaluated by the attending. This discussion often occurred in the team rooms, with the attending leading the discussion with the trainees.

Data Collection

A multidisciplinary team, including clinicians (eg, physicians, nurses), nonclinicians (eg, qualitative researchers, social scientists), and healthcare engineers, conducted the observations. We observed preround activities of interns and residents before arrival of the attending (7:00 AM - 8:30 AM), followed by morning rounds with the entire team, and afternoon work that included senior residents, interns, and students.

To capture multiple aspects of the diagnostic process, we collected data using field notes modeled on components of the National Academy of Science model for diagnosis (Appendix).^1,15 This model encompasses phases of the diagnostic process (eg, data gathering, integration, formulation of a working diagnosis, treatment delivery, and outcomes) and the work system (team members, organization, technology and tools, physical environment, tasks).

Focus Groups and Interviews

At the end of weekly observations, we conducted focus groups with the residents and one-on- one interviews with the attendings. Focus groups with the residents were conducted to encourage a group discussion about the diagnostic process. Separate interviews with the attendings were performed to ensure that power differentials did not influence discussions. During focus groups, we specifically asked about challenges and possible solutions to improve diagnosis. Experienced qualitative methodologists (J.F., M.H., M.Q.) used semistructured interview guides for discussions (Appendix).

Data Analysis

After aggregating and reading the data, three reviewers (V.C., S.K., S.S.) began inductive analysis by handwriting notes and initial reflective thoughts to create preliminary codes. Multiple team members then reread the original field notes and the focus group/interview data to refine the preliminary codes and develop additional codes. Next, relationships between codes were identified and used to develop key themes. Triangulation of data collected from observations and interview/focus group sessions was carried out to compare data that we surmised with data that were verbalized by the team. The developed themes were discussed as a group to ensure consistency of major findings.

Ethical and Regulatory Oversight

This study was reviewed and approved by the Institutional Review Boards at the University of Michigan Health System (HUM-00106657) and the VA Ann Arbor Healthcare System (1-2016-010040).

Four teaching teams (4 attendings, 4 senior residents, 9 interns, and 14 medical students) were observed over 33 distinct shifts and 168 hours. Observations included morning rounds (96 h), postround call days (52 h), and postround non-call days (20 h). Morning rounds lasted an average of 127 min (range: 48-232 min) and included an average of 9 patients (range: 4-16 patients).

Themes Regarding the Diagnostic Process

We identified the following 4 primary themes related to the diagnostic process in teaching hospitals: (1) diagnosis is a social phenomenon; (2) data necessary to make diagnoses are fragmented; (3) distractions undermine the diagnostic process; and (4) time pressures interfere with diagnostic decision making (Appendix Table 1).

(1) Diagnosis is a Social Phenomenon.

Team members viewed the process of diagnosis as a social exchange of facts, findings, and strategies within a defined structure. The opportunity to discuss impressions with others was valued as a means to share, test, and process assumptions.

“Rounds are the most important part of the process. That is where we make most decisions in a collective, collaborative way with the attending present. We bounce ideas off each other.” (Intern)

Typical of social processes, variations based on time of day and schedule were observed. For instance, during call days, learners gathered data and formed working diagnosis and treatment plans with minimal attending interaction. This separation of roles and responsibilities introduced a hierarchy within diagnosis as follows:

“The interns would not call me first; they would talk to the senior resident and then if the senior thought he should chat with me, then they would call. But for the most part, they gather information and come up with the plan.” (Attending).

The work system was suited to facilitate social interactions. For instance, designated rooms (with team members informally assigned to a computer) provided physical proximity of the resident to interns and medical students. In this space, numerous informal discussions between team members (eg, “What do you think about this test?” “I’m not sure what to do about this finding.” “Should I call a [consult] on this patient?”) were observed. Although proximity to each other was viewed as beneficial, dangers to the social nature of diagnosis in the form of anchoring (ie, a cognitive bias where emphasis is placed on the first piece of data)¹⁶ were also mentioned. Similarly, the paradox associated with social proof (ie, the pressure to assume conformity within a group) was also observed as disagreement between team members and attendings rarely occurred during observations.

“I mean, they’re the attending, right? It’s hard to argue with them when they want a test or something done. When I do push back, it’s rare that others will support me–so it’s usually me and the attending.” (Resident)

“I would push back if I think it’s really bad for the patient or could cause harm–but the truth is, it doesn’t happen much.” (Intern)

(2) Data Necessary to Make Diagnoses are Fragmented

Team members universally cited fragmentation in data delivery, retrieval, and processing as a barrier to diagnosis. Team members indicated that test results might not be looked at or acted upon in a timely manner, and participants pointed to the electronic medical record as a source of this challenge.

“Before I knew about [the app for Epic], I would literally sit on the computer to get all the information we would need on rounds. Its key to making decisions. We often say we will do something, only to find the test result doesn’t support it–and then we’re back to square 1.” (Intern)

Information used by teams came from myriad sources (eg, patients, family members, electronic records) and from various settings (eg, emergency department, patient rooms, discussions with consultants). Additionally, test results often appeared without warning. Thus, availability of information was poorly aligned with clinical duties.

“They (the lab) will call us when a blood culture is positive or something is off. That is very helpful but it often comes later in the day, when we’re done with rounds.” (Resident)

The work system was highlighted as a key contributor to data fragmentation. Peculiarities of our electronic medical record (EMR) and how data were collected, stored, or presented were described as “frustrating,” and “unsafe,” by team members. Correspondingly, we frequently observed interns asking for assistance for tasks such as ordering tests or finding information despite being “trained” to use the EMR.

“People have to learn how to filter, how to recognize the most important points and link data streams together in terms of causality. But we assume they know where to find that information. It’s actually a very hard thing to do, for both the house staff and me.” (Attending)

(3) Distractions Undermine the Diagnostic Process

Distractions often created cognitive difficulties. For example, ambient noise and interruptions from neighbors working on other teams were cited as barriers to diagnosis. In addition, we observed several team members using headphones to drown out ambient noise while working on the computer.

“I know I shouldn’t do it (wear headphones), but I have no other way of turning down the noise so I can concentrate.” (Intern)

Similarly, the unpredictable nature and the volume of pages often interrupted thinking about diagnosis.

“Sometimes the pager just goes off all the time and (after making sure its not an urgent issue), I will just ignore it for a bit, especially if I am in the middle of something. It would be great if I could finish my thought process knowing I would not be interrupted.” (Resident)

To mitigate this problem, 1 attending described how he would proactively seek out nurses caring for his patients to “head off” questions (eg, “I will renew the restraints and medications this morning,” and “Is there anything you need in terms of orders for this patient that I can take care of now?”) that might lead to pages. Another resident described his approach as follows:

“I make it a point to tell the nurses where I will be hanging out and where they can find me if they have any questions. I tell them to come talk to me rather than page me since that will be less distracting.” (Resident).

Most of the interns described documentation work such as writing admission and progress notes in negative terms (“an academic exercise,” “part of the billing activity”). However, in the context of interruptions, some described this as helpful.

“The most valuable part of the thinking process was writing the assessment and plan because that’s actually my schema for all problems. It literally is the only time where I can sit and collect my thoughts to formulate a diagnosis and plan.” (Intern)

(4) Time Pressures Interfere With Diagnostic Decision Making

All team members spoke about the challenge of finding time for diagnosis during the workday. Often, they had to skip learning sessions for this purpose.

“They tell us we should go to morning report or noon conference but when I’m running around trying to get things done. I hate having to choose between my education and doing what’s best for the patient–but that’s often what it comes down to.” (Intern)

When specifically asked whether setting aside dedicated time to specifically review and formulate diagnoses would be valuable, respondents were uniformly enthusiastic. Team members described attentional conflicts as being the worst when “cross covering” other teams on call days, as their patient load effectively doubled during this time. Of note, cross-covering occurred when teams were also on call—and thus took them away from important diagnostic activities such as data gathering or synthesis for patients they were admitting.

“If you were to ever design a system where errors were likely–this is how you would design it: take a team with little supervision, double their patient load, keep them busy with new challenging cases and then ask questions about patients they know little about.” (Resident)

Although diagnostic errors have been called “the next frontier for patient safety,”¹⁷ little is known about the process, barriers, and facilitators to diagnosis in teaching hospitals. In this focused ethnography conducted at 2 academic medical centers, we identified multiple cognitive and system-level challenges and potential strategies to improve diagnosis from trainees engaged in this activity. Key themes identified by those we observed included the social nature of diagnosis, fragmented information delivery, constant distractions and interruptions, and time pressures. In turn, these insights allow us to generate strategies that can be applied to improve the diagnostic process in teaching hospitals.

Our study underscores the importance of social interactions in diagnosis. In contrast, most of the interventions to prevent diagnostic errors target individual providers through practices such as metacognition and “thinking about thinking.”^18-20 These interventions are based on Daniel Kahnemann’s work on dual thought process. Type 1 thought processes are fast, subconscious, reflexive, largely intuitive, and more vulnerable to error. In contrast, Type 2 processes are slower, deliberate, analytic, and less prone to error.²¹ Although an individual’s Type 2 thought capacity is limited, a major goal of cognitive interventions is to encourage Type 2 over Type 1 thinking, an approach termed “de-biasing.”^22-24 Unfortunately, cognitive interventions testing such approaches have suffered mixed results–perhaps because of lack of focus on collective wisdom or group thinking, which may be key to diagnosis from our findings.^9,25 In this sense, morning rounds were a social gathering used to strategize and develop care plans, but with limited time to think about diagnosis.²⁶ Introduction of defined periods for individuals to engage in diagnostic activities such as de-biasing (ie, asking “what else could this be)²⁷ before or after rounds may provide an opportunity for reflection and improving diagnosis. In addition, embedding tools such as diagnosis expanders and checklists within these defined time slots^28,29 may prove to be useful in reflecting on diagnosis and preventing diagnostic errors.

An unexpected yet important finding from this study were the challenges posed by distractions and the physical environment. Potentially maladaptive workarounds to these interruptions included use of headphones; more productive strategies included updating nurses with plans to avert pages and creating a list of activities to ensure that key tasks were not forgotten.^30,31 Applying lessons from aviation, a focused effort to limit distractions during key portions of the day, might be worth considering for diagnostic safety.³² Similarly, improving the environment in which diagnosis occurs—including creating spaces that are quiet, orderly, and optimized for thinking—may be valuable.³³Our study has limitations. First, our findings are limited to direct observations; we are thus unable to comment on how unobserved aspects of care (eg, cognitive processes) might have influenced our findings. Our observations of clinical care might also have introduced a Hawthorne effect. However, because we were closely integrated with teams and conducted focus groups to corroborate our assessments, we believe that this was not the case. Second, we did not identify diagnostic errors or link processes we observed to errors. Third, our approach is limited to 2 teaching centers, thereby limiting the generalizability of findings. Relatedly, we were only able to conduct observations during weekdays; differences in weekend and night resources might affect our insights.

The cognitive and system-based barriers faced by clinicians in teaching hospitals suggest that new methods to improve diagnosis are needed. Future interventions such as defined “time-outs” for diagnosis, strategies focused on limiting distractions, and methods to improve communication between team members are novel and have parallels in other industries. As challenges to quantify diagnostic errors abound,³⁴ improving cognitive- and system-based factors via reflection through communication, concentration, and organization is necessary to improve medical decision making in academic medical centers.

Disclosures

None declared for all coauthors.

Funding

This project was supported by grant number P30HS024385 from the Agency for Healthcare Research and Quality. The funding source played no role in study design, data acquisition, analysis or decision to report these data. Dr. Chopra is supported by a career development award from the Agency of Healthcare Research and Quality (1-K08-HS022835-01). Dr. Krein is supported by a VA Health Services Research and Development Research Career Scientist Award (RCS 11-222). Dr. Singh is partially supported by Houston VA HSR&D Center for Innovations in Quality, Effectiveness and Safety (CIN 13-413). The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality or the Department of Veterans Affairs.

Diagnostic error—defined as a failure to establish an accurate and timely explanation of the patient’s health problem—is an important source of patient harm.¹ Data suggest that all patients will experience at least 1 diagnostic error in their lifetime.^2-4 Not surprisingly, diagnostic errors are among the leading categories of paid malpractice claims in the United States.⁵

Despite diagnostic errors being morbid and sometimes deadly in the hospital,^6,7 little is known about how residents and learners approach diagnostic decision making. Errors in diagnosis are believed to stem from cognitive or system failures,⁸ with errors in cognition believed to occur due to rapid, reflexive thinking operating in the absence of a more analytical, deliberate process. System-based problems (eg, lack of expert availability, technology barriers, and access to data) have also been cited as contributors.⁹ However, whether and how these apply to trainees is not known.

Therefore, we conducted a focused ethnography of inpatient medicine teams (ie, attendings, residents, interns, and medical students) in 2 affiliated teaching hospitals, aiming to (a) observe the process of diagnosis by trainees and (b) identify methods to improve the diagnostic process and prevent errors.

We designed a multimethod, focused ethnographic study to examine diagnostic decision making in hospital settings.^10,11 In contrast to anthropologic ethnographies that study entire fields using open-ended questions, our study was designed to examine the process of diagnosis from the perspective of clinicians engaged in this activity.¹¹ This approach allowed us to capture diagnostic decisions and cognitive and system-based factors in a manner currently lacking in the literature.¹²

Setting and Participants

Between January 2016 and May 2016, we observed the members of four inpatient internal medicine teaching teams at 2 affiliated teaching hospitals. We purposefully selected teaching teams for observation because they are the primary model of care in academic settings and we have expertise in carrying out similar studies.^13,14 Teaching teams typically consisted of a medical attending (senior-level physician), 1 senior resident (a second- or third-year postgraduate trainee), two interns (a trainee in their first postgraduate year), and two to four  medical students. Teams were selected at random using existing schedules and followed Monday to Friday so as to permit observation of work on call and noncall days. Owing to manpower limitations, weekend and night shifts were not observed. However, overnight events were captured during morning rounds.

Most of the teams began rounds at 8:30 AM. Typically, rounds lasted for 90–120 min and concluded with a recap (ie, “running the list”) with a review of explicit plans for patients after they had been evaluated by the attending. This discussion often occurred in the team rooms, with the attending leading the discussion with the trainees.

Data Collection

A multidisciplinary team, including clinicians (eg, physicians, nurses), nonclinicians (eg, qualitative researchers, social scientists), and healthcare engineers, conducted the observations. We observed preround activities of interns and residents before arrival of the attending (7:00 AM - 8:30 AM), followed by morning rounds with the entire team, and afternoon work that included senior residents, interns, and students.

To capture multiple aspects of the diagnostic process, we collected data using field notes modeled on components of the National Academy of Science model for diagnosis (Appendix).^1,15 This model encompasses phases of the diagnostic process (eg, data gathering, integration, formulation of a working diagnosis, treatment delivery, and outcomes) and the work system (team members, organization, technology and tools, physical environment, tasks).

Focus Groups and Interviews

At the end of weekly observations, we conducted focus groups with the residents and one-on- one interviews with the attendings. Focus groups with the residents were conducted to encourage a group discussion about the diagnostic process. Separate interviews with the attendings were performed to ensure that power differentials did not influence discussions. During focus groups, we specifically asked about challenges and possible solutions to improve diagnosis. Experienced qualitative methodologists (J.F., M.H., M.Q.) used semistructured interview guides for discussions (Appendix).

Data Analysis

After aggregating and reading the data, three reviewers (V.C., S.K., S.S.) began inductive analysis by handwriting notes and initial reflective thoughts to create preliminary codes. Multiple team members then reread the original field notes and the focus group/interview data to refine the preliminary codes and develop additional codes. Next, relationships between codes were identified and used to develop key themes. Triangulation of data collected from observations and interview/focus group sessions was carried out to compare data that we surmised with data that were verbalized by the team. The developed themes were discussed as a group to ensure consistency of major findings.

Ethical and Regulatory Oversight

This study was reviewed and approved by the Institutional Review Boards at the University of Michigan Health System (HUM-00106657) and the VA Ann Arbor Healthcare System (1-2016-010040).

Four teaching teams (4 attendings, 4 senior residents, 9 interns, and 14 medical students) were observed over 33 distinct shifts and 168 hours. Observations included morning rounds (96 h), postround call days (52 h), and postround non-call days (20 h). Morning rounds lasted an average of 127 min (range: 48-232 min) and included an average of 9 patients (range: 4-16 patients).

Themes Regarding the Diagnostic Process

We identified the following 4 primary themes related to the diagnostic process in teaching hospitals: (1) diagnosis is a social phenomenon; (2) data necessary to make diagnoses are fragmented; (3) distractions undermine the diagnostic process; and (4) time pressures interfere with diagnostic decision making (Appendix Table 1).

(1) Diagnosis is a Social Phenomenon.

Team members viewed the process of diagnosis as a social exchange of facts, findings, and strategies within a defined structure. The opportunity to discuss impressions with others was valued as a means to share, test, and process assumptions.

“Rounds are the most important part of the process. That is where we make most decisions in a collective, collaborative way with the attending present. We bounce ideas off each other.” (Intern)

Typical of social processes, variations based on time of day and schedule were observed. For instance, during call days, learners gathered data and formed working diagnosis and treatment plans with minimal attending interaction. This separation of roles and responsibilities introduced a hierarchy within diagnosis as follows:

“The interns would not call me first; they would talk to the senior resident and then if the senior thought he should chat with me, then they would call. But for the most part, they gather information and come up with the plan.” (Attending).

The work system was suited to facilitate social interactions. For instance, designated rooms (with team members informally assigned to a computer) provided physical proximity of the resident to interns and medical students. In this space, numerous informal discussions between team members (eg, “What do you think about this test?” “I’m not sure what to do about this finding.” “Should I call a [consult] on this patient?”) were observed. Although proximity to each other was viewed as beneficial, dangers to the social nature of diagnosis in the form of anchoring (ie, a cognitive bias where emphasis is placed on the first piece of data)¹⁶ were also mentioned. Similarly, the paradox associated with social proof (ie, the pressure to assume conformity within a group) was also observed as disagreement between team members and attendings rarely occurred during observations.

“I mean, they’re the attending, right? It’s hard to argue with them when they want a test or something done. When I do push back, it’s rare that others will support me–so it’s usually me and the attending.” (Resident)

“I would push back if I think it’s really bad for the patient or could cause harm–but the truth is, it doesn’t happen much.” (Intern)

(2) Data Necessary to Make Diagnoses are Fragmented

Team members universally cited fragmentation in data delivery, retrieval, and processing as a barrier to diagnosis. Team members indicated that test results might not be looked at or acted upon in a timely manner, and participants pointed to the electronic medical record as a source of this challenge.

“Before I knew about [the app for Epic], I would literally sit on the computer to get all the information we would need on rounds. Its key to making decisions. We often say we will do something, only to find the test result doesn’t support it–and then we’re back to square 1.” (Intern)

Information used by teams came from myriad sources (eg, patients, family members, electronic records) and from various settings (eg, emergency department, patient rooms, discussions with consultants). Additionally, test results often appeared without warning. Thus, availability of information was poorly aligned with clinical duties.

“They (the lab) will call us when a blood culture is positive or something is off. That is very helpful but it often comes later in the day, when we’re done with rounds.” (Resident)

The work system was highlighted as a key contributor to data fragmentation. Peculiarities of our electronic medical record (EMR) and how data were collected, stored, or presented were described as “frustrating,” and “unsafe,” by team members. Correspondingly, we frequently observed interns asking for assistance for tasks such as ordering tests or finding information despite being “trained” to use the EMR.

“People have to learn how to filter, how to recognize the most important points and link data streams together in terms of causality. But we assume they know where to find that information. It’s actually a very hard thing to do, for both the house staff and me.” (Attending)

(3) Distractions Undermine the Diagnostic Process

Distractions often created cognitive difficulties. For example, ambient noise and interruptions from neighbors working on other teams were cited as barriers to diagnosis. In addition, we observed several team members using headphones to drown out ambient noise while working on the computer.

“I know I shouldn’t do it (wear headphones), but I have no other way of turning down the noise so I can concentrate.” (Intern)

Similarly, the unpredictable nature and the volume of pages often interrupted thinking about diagnosis.

“Sometimes the pager just goes off all the time and (after making sure its not an urgent issue), I will just ignore it for a bit, especially if I am in the middle of something. It would be great if I could finish my thought process knowing I would not be interrupted.” (Resident)

To mitigate this problem, 1 attending described how he would proactively seek out nurses caring for his patients to “head off” questions (eg, “I will renew the restraints and medications this morning,” and “Is there anything you need in terms of orders for this patient that I can take care of now?”) that might lead to pages. Another resident described his approach as follows:

“I make it a point to tell the nurses where I will be hanging out and where they can find me if they have any questions. I tell them to come talk to me rather than page me since that will be less distracting.” (Resident).

Most of the interns described documentation work such as writing admission and progress notes in negative terms (“an academic exercise,” “part of the billing activity”). However, in the context of interruptions, some described this as helpful.

“The most valuable part of the thinking process was writing the assessment and plan because that’s actually my schema for all problems. It literally is the only time where I can sit and collect my thoughts to formulate a diagnosis and plan.” (Intern)

(4) Time Pressures Interfere With Diagnostic Decision Making

All team members spoke about the challenge of finding time for diagnosis during the workday. Often, they had to skip learning sessions for this purpose.

“They tell us we should go to morning report or noon conference but when I’m running around trying to get things done. I hate having to choose between my education and doing what’s best for the patient–but that’s often what it comes down to.” (Intern)

When specifically asked whether setting aside dedicated time to specifically review and formulate diagnoses would be valuable, respondents were uniformly enthusiastic. Team members described attentional conflicts as being the worst when “cross covering” other teams on call days, as their patient load effectively doubled during this time. Of note, cross-covering occurred when teams were also on call—and thus took them away from important diagnostic activities such as data gathering or synthesis for patients they were admitting.

“If you were to ever design a system where errors were likely–this is how you would design it: take a team with little supervision, double their patient load, keep them busy with new challenging cases and then ask questions about patients they know little about.” (Resident)

Although diagnostic errors have been called “the next frontier for patient safety,”¹⁷ little is known about the process, barriers, and facilitators to diagnosis in teaching hospitals. In this focused ethnography conducted at 2 academic medical centers, we identified multiple cognitive and system-level challenges and potential strategies to improve diagnosis from trainees engaged in this activity. Key themes identified by those we observed included the social nature of diagnosis, fragmented information delivery, constant distractions and interruptions, and time pressures. In turn, these insights allow us to generate strategies that can be applied to improve the diagnostic process in teaching hospitals.

Our study underscores the importance of social interactions in diagnosis. In contrast, most of the interventions to prevent diagnostic errors target individual providers through practices such as metacognition and “thinking about thinking.”^18-20 These interventions are based on Daniel Kahnemann’s work on dual thought process. Type 1 thought processes are fast, subconscious, reflexive, largely intuitive, and more vulnerable to error. In contrast, Type 2 processes are slower, deliberate, analytic, and less prone to error.²¹ Although an individual’s Type 2 thought capacity is limited, a major goal of cognitive interventions is to encourage Type 2 over Type 1 thinking, an approach termed “de-biasing.”^22-24 Unfortunately, cognitive interventions testing such approaches have suffered mixed results–perhaps because of lack of focus on collective wisdom or group thinking, which may be key to diagnosis from our findings.^9,25 In this sense, morning rounds were a social gathering used to strategize and develop care plans, but with limited time to think about diagnosis.²⁶ Introduction of defined periods for individuals to engage in diagnostic activities such as de-biasing (ie, asking “what else could this be)²⁷ before or after rounds may provide an opportunity for reflection and improving diagnosis. In addition, embedding tools such as diagnosis expanders and checklists within these defined time slots^28,29 may prove to be useful in reflecting on diagnosis and preventing diagnostic errors.

An unexpected yet important finding from this study were the challenges posed by distractions and the physical environment. Potentially maladaptive workarounds to these interruptions included use of headphones; more productive strategies included updating nurses with plans to avert pages and creating a list of activities to ensure that key tasks were not forgotten.^30,31 Applying lessons from aviation, a focused effort to limit distractions during key portions of the day, might be worth considering for diagnostic safety.³² Similarly, improving the environment in which diagnosis occurs—including creating spaces that are quiet, orderly, and optimized for thinking—may be valuable.³³Our study has limitations. First, our findings are limited to direct observations; we are thus unable to comment on how unobserved aspects of care (eg, cognitive processes) might have influenced our findings. Our observations of clinical care might also have introduced a Hawthorne effect. However, because we were closely integrated with teams and conducted focus groups to corroborate our assessments, we believe that this was not the case. Second, we did not identify diagnostic errors or link processes we observed to errors. Third, our approach is limited to 2 teaching centers, thereby limiting the generalizability of findings. Relatedly, we were only able to conduct observations during weekdays; differences in weekend and night resources might affect our insights.

The cognitive and system-based barriers faced by clinicians in teaching hospitals suggest that new methods to improve diagnosis are needed. Future interventions such as defined “time-outs” for diagnosis, strategies focused on limiting distractions, and methods to improve communication between team members are novel and have parallels in other industries. As challenges to quantify diagnostic errors abound,³⁴ improving cognitive- and system-based factors via reflection through communication, concentration, and organization is necessary to improve medical decision making in academic medical centers.

Disclosures

None declared for all coauthors.

Funding

This project was supported by grant number P30HS024385 from the Agency for Healthcare Research and Quality. The funding source played no role in study design, data acquisition, analysis or decision to report these data. Dr. Chopra is supported by a career development award from the Agency of Healthcare Research and Quality (1-K08-HS022835-01). Dr. Krein is supported by a VA Health Services Research and Development Research Career Scientist Award (RCS 11-222). Dr. Singh is partially supported by Houston VA HSR&D Center for Innovations in Quality, Effectiveness and Safety (CIN 13-413). The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality or the Department of Veterans Affairs.

Diagnostic error—defined as a failure to establish an accurate and timely explanation of the patient’s health problem—is an important source of patient harm.¹ Data suggest that all patients will experience at least 1 diagnostic error in their lifetime.^2-4 Not surprisingly, diagnostic errors are among the leading categories of paid malpractice claims in the United States.⁵

Despite diagnostic errors being morbid and sometimes deadly in the hospital,^6,7 little is known about how residents and learners approach diagnostic decision making. Errors in diagnosis are believed to stem from cognitive or system failures,⁸ with errors in cognition believed to occur due to rapid, reflexive thinking operating in the absence of a more analytical, deliberate process. System-based problems (eg, lack of expert availability, technology barriers, and access to data) have also been cited as contributors.⁹ However, whether and how these apply to trainees is not known.

Therefore, we conducted a focused ethnography of inpatient medicine teams (ie, attendings, residents, interns, and medical students) in 2 affiliated teaching hospitals, aiming to (a) observe the process of diagnosis by trainees and (b) identify methods to improve the diagnostic process and prevent errors.

We designed a multimethod, focused ethnographic study to examine diagnostic decision making in hospital settings.^10,11 In contrast to anthropologic ethnographies that study entire fields using open-ended questions, our study was designed to examine the process of diagnosis from the perspective of clinicians engaged in this activity.¹¹ This approach allowed us to capture diagnostic decisions and cognitive and system-based factors in a manner currently lacking in the literature.¹²

Setting and Participants

Between January 2016 and May 2016, we observed the members of four inpatient internal medicine teaching teams at 2 affiliated teaching hospitals. We purposefully selected teaching teams for observation because they are the primary model of care in academic settings and we have expertise in carrying out similar studies.^13,14 Teaching teams typically consisted of a medical attending (senior-level physician), 1 senior resident (a second- or third-year postgraduate trainee), two interns (a trainee in their first postgraduate year), and two to four  medical students. Teams were selected at random using existing schedules and followed Monday to Friday so as to permit observation of work on call and noncall days. Owing to manpower limitations, weekend and night shifts were not observed. However, overnight events were captured during morning rounds.

Most of the teams began rounds at 8:30 AM. Typically, rounds lasted for 90–120 min and concluded with a recap (ie, “running the list”) with a review of explicit plans for patients after they had been evaluated by the attending. This discussion often occurred in the team rooms, with the attending leading the discussion with the trainees.

Data Collection

A multidisciplinary team, including clinicians (eg, physicians, nurses), nonclinicians (eg, qualitative researchers, social scientists), and healthcare engineers, conducted the observations. We observed preround activities of interns and residents before arrival of the attending (7:00 AM - 8:30 AM), followed by morning rounds with the entire team, and afternoon work that included senior residents, interns, and students.

To capture multiple aspects of the diagnostic process, we collected data using field notes modeled on components of the National Academy of Science model for diagnosis (Appendix).^1,15 This model encompasses phases of the diagnostic process (eg, data gathering, integration, formulation of a working diagnosis, treatment delivery, and outcomes) and the work system (team members, organization, technology and tools, physical environment, tasks).

Focus Groups and Interviews

At the end of weekly observations, we conducted focus groups with the residents and one-on- one interviews with the attendings. Focus groups with the residents were conducted to encourage a group discussion about the diagnostic process. Separate interviews with the attendings were performed to ensure that power differentials did not influence discussions. During focus groups, we specifically asked about challenges and possible solutions to improve diagnosis. Experienced qualitative methodologists (J.F., M.H., M.Q.) used semistructured interview guides for discussions (Appendix).

Data Analysis

After aggregating and reading the data, three reviewers (V.C., S.K., S.S.) began inductive analysis by handwriting notes and initial reflective thoughts to create preliminary codes. Multiple team members then reread the original field notes and the focus group/interview data to refine the preliminary codes and develop additional codes. Next, relationships between codes were identified and used to develop key themes. Triangulation of data collected from observations and interview/focus group sessions was carried out to compare data that we surmised with data that were verbalized by the team. The developed themes were discussed as a group to ensure consistency of major findings.

Ethical and Regulatory Oversight

This study was reviewed and approved by the Institutional Review Boards at the University of Michigan Health System (HUM-00106657) and the VA Ann Arbor Healthcare System (1-2016-010040).

Four teaching teams (4 attendings, 4 senior residents, 9 interns, and 14 medical students) were observed over 33 distinct shifts and 168 hours. Observations included morning rounds (96 h), postround call days (52 h), and postround non-call days (20 h). Morning rounds lasted an average of 127 min (range: 48-232 min) and included an average of 9 patients (range: 4-16 patients).

Themes Regarding the Diagnostic Process

We identified the following 4 primary themes related to the diagnostic process in teaching hospitals: (1) diagnosis is a social phenomenon; (2) data necessary to make diagnoses are fragmented; (3) distractions undermine the diagnostic process; and (4) time pressures interfere with diagnostic decision making (Appendix Table 1).

(1) Diagnosis is a Social Phenomenon.

Team members viewed the process of diagnosis as a social exchange of facts, findings, and strategies within a defined structure. The opportunity to discuss impressions with others was valued as a means to share, test, and process assumptions.

“Rounds are the most important part of the process. That is where we make most decisions in a collective, collaborative way with the attending present. We bounce ideas off each other.” (Intern)

Typical of social processes, variations based on time of day and schedule were observed. For instance, during call days, learners gathered data and formed working diagnosis and treatment plans with minimal attending interaction. This separation of roles and responsibilities introduced a hierarchy within diagnosis as follows:

“The interns would not call me first; they would talk to the senior resident and then if the senior thought he should chat with me, then they would call. But for the most part, they gather information and come up with the plan.” (Attending).

The work system was suited to facilitate social interactions. For instance, designated rooms (with team members informally assigned to a computer) provided physical proximity of the resident to interns and medical students. In this space, numerous informal discussions between team members (eg, “What do you think about this test?” “I’m not sure what to do about this finding.” “Should I call a [consult] on this patient?”) were observed. Although proximity to each other was viewed as beneficial, dangers to the social nature of diagnosis in the form of anchoring (ie, a cognitive bias where emphasis is placed on the first piece of data)¹⁶ were also mentioned. Similarly, the paradox associated with social proof (ie, the pressure to assume conformity within a group) was also observed as disagreement between team members and attendings rarely occurred during observations.

“I mean, they’re the attending, right? It’s hard to argue with them when they want a test or something done. When I do push back, it’s rare that others will support me–so it’s usually me and the attending.” (Resident)

“I would push back if I think it’s really bad for the patient or could cause harm–but the truth is, it doesn’t happen much.” (Intern)

(2) Data Necessary to Make Diagnoses are Fragmented

Team members universally cited fragmentation in data delivery, retrieval, and processing as a barrier to diagnosis. Team members indicated that test results might not be looked at or acted upon in a timely manner, and participants pointed to the electronic medical record as a source of this challenge.

“Before I knew about [the app for Epic], I would literally sit on the computer to get all the information we would need on rounds. Its key to making decisions. We often say we will do something, only to find the test result doesn’t support it–and then we’re back to square 1.” (Intern)

Information used by teams came from myriad sources (eg, patients, family members, electronic records) and from various settings (eg, emergency department, patient rooms, discussions with consultants). Additionally, test results often appeared without warning. Thus, availability of information was poorly aligned with clinical duties.

“They (the lab) will call us when a blood culture is positive or something is off. That is very helpful but it often comes later in the day, when we’re done with rounds.” (Resident)

The work system was highlighted as a key contributor to data fragmentation. Peculiarities of our electronic medical record (EMR) and how data were collected, stored, or presented were described as “frustrating,” and “unsafe,” by team members. Correspondingly, we frequently observed interns asking for assistance for tasks such as ordering tests or finding information despite being “trained” to use the EMR.

“People have to learn how to filter, how to recognize the most important points and link data streams together in terms of causality. But we assume they know where to find that information. It’s actually a very hard thing to do, for both the house staff and me.” (Attending)

(3) Distractions Undermine the Diagnostic Process

Distractions often created cognitive difficulties. For example, ambient noise and interruptions from neighbors working on other teams were cited as barriers to diagnosis. In addition, we observed several team members using headphones to drown out ambient noise while working on the computer.

“I know I shouldn’t do it (wear headphones), but I have no other way of turning down the noise so I can concentrate.” (Intern)

Similarly, the unpredictable nature and the volume of pages often interrupted thinking about diagnosis.

“Sometimes the pager just goes off all the time and (after making sure its not an urgent issue), I will just ignore it for a bit, especially if I am in the middle of something. It would be great if I could finish my thought process knowing I would not be interrupted.” (Resident)

To mitigate this problem, 1 attending described how he would proactively seek out nurses caring for his patients to “head off” questions (eg, “I will renew the restraints and medications this morning,” and “Is there anything you need in terms of orders for this patient that I can take care of now?”) that might lead to pages. Another resident described his approach as follows:

“I make it a point to tell the nurses where I will be hanging out and where they can find me if they have any questions. I tell them to come talk to me rather than page me since that will be less distracting.” (Resident).

Most of the interns described documentation work such as writing admission and progress notes in negative terms (“an academic exercise,” “part of the billing activity”). However, in the context of interruptions, some described this as helpful.

“The most valuable part of the thinking process was writing the assessment and plan because that’s actually my schema for all problems. It literally is the only time where I can sit and collect my thoughts to formulate a diagnosis and plan.” (Intern)

(4) Time Pressures Interfere With Diagnostic Decision Making

All team members spoke about the challenge of finding time for diagnosis during the workday. Often, they had to skip learning sessions for this purpose.

“They tell us we should go to morning report or noon conference but when I’m running around trying to get things done. I hate having to choose between my education and doing what’s best for the patient–but that’s often what it comes down to.” (Intern)

When specifically asked whether setting aside dedicated time to specifically review and formulate diagnoses would be valuable, respondents were uniformly enthusiastic. Team members described attentional conflicts as being the worst when “cross covering” other teams on call days, as their patient load effectively doubled during this time. Of note, cross-covering occurred when teams were also on call—and thus took them away from important diagnostic activities such as data gathering or synthesis for patients they were admitting.

“If you were to ever design a system where errors were likely–this is how you would design it: take a team with little supervision, double their patient load, keep them busy with new challenging cases and then ask questions about patients they know little about.” (Resident)

Although diagnostic errors have been called “the next frontier for patient safety,”¹⁷ little is known about the process, barriers, and facilitators to diagnosis in teaching hospitals. In this focused ethnography conducted at 2 academic medical centers, we identified multiple cognitive and system-level challenges and potential strategies to improve diagnosis from trainees engaged in this activity. Key themes identified by those we observed included the social nature of diagnosis, fragmented information delivery, constant distractions and interruptions, and time pressures. In turn, these insights allow us to generate strategies that can be applied to improve the diagnostic process in teaching hospitals.

Our study underscores the importance of social interactions in diagnosis. In contrast, most of the interventions to prevent diagnostic errors target individual providers through practices such as metacognition and “thinking about thinking.”^18-20 These interventions are based on Daniel Kahnemann’s work on dual thought process. Type 1 thought processes are fast, subconscious, reflexive, largely intuitive, and more vulnerable to error. In contrast, Type 2 processes are slower, deliberate, analytic, and less prone to error.²¹ Although an individual’s Type 2 thought capacity is limited, a major goal of cognitive interventions is to encourage Type 2 over Type 1 thinking, an approach termed “de-biasing.”^22-24 Unfortunately, cognitive interventions testing such approaches have suffered mixed results–perhaps because of lack of focus on collective wisdom or group thinking, which may be key to diagnosis from our findings.^9,25 In this sense, morning rounds were a social gathering used to strategize and develop care plans, but with limited time to think about diagnosis.²⁶ Introduction of defined periods for individuals to engage in diagnostic activities such as de-biasing (ie, asking “what else could this be)²⁷ before or after rounds may provide an opportunity for reflection and improving diagnosis. In addition, embedding tools such as diagnosis expanders and checklists within these defined time slots^28,29 may prove to be useful in reflecting on diagnosis and preventing diagnostic errors.

An unexpected yet important finding from this study were the challenges posed by distractions and the physical environment. Potentially maladaptive workarounds to these interruptions included use of headphones; more productive strategies included updating nurses with plans to avert pages and creating a list of activities to ensure that key tasks were not forgotten.^30,31 Applying lessons from aviation, a focused effort to limit distractions during key portions of the day, might be worth considering for diagnostic safety.³² Similarly, improving the environment in which diagnosis occurs—including creating spaces that are quiet, orderly, and optimized for thinking—may be valuable.³³Our study has limitations. First, our findings are limited to direct observations; we are thus unable to comment on how unobserved aspects of care (eg, cognitive processes) might have influenced our findings. Our observations of clinical care might also have introduced a Hawthorne effect. However, because we were closely integrated with teams and conducted focus groups to corroborate our assessments, we believe that this was not the case. Second, we did not identify diagnostic errors or link processes we observed to errors. Third, our approach is limited to 2 teaching centers, thereby limiting the generalizability of findings. Relatedly, we were only able to conduct observations during weekdays; differences in weekend and night resources might affect our insights.

The cognitive and system-based barriers faced by clinicians in teaching hospitals suggest that new methods to improve diagnosis are needed. Future interventions such as defined “time-outs” for diagnosis, strategies focused on limiting distractions, and methods to improve communication between team members are novel and have parallels in other industries. As challenges to quantify diagnostic errors abound,³⁴ improving cognitive- and system-based factors via reflection through communication, concentration, and organization is necessary to improve medical decision making in academic medical centers.

Disclosures

None declared for all coauthors.

Funding

This project was supported by grant number P30HS024385 from the Agency for Healthcare Research and Quality. The funding source played no role in study design, data acquisition, analysis or decision to report these data. Dr. Chopra is supported by a career development award from the Agency of Healthcare Research and Quality (1-K08-HS022835-01). Dr. Krein is supported by a VA Health Services Research and Development Research Career Scientist Award (RCS 11-222). Dr. Singh is partially supported by Houston VA HSR&D Center for Innovations in Quality, Effectiveness and Safety (CIN 13-413). The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality or the Department of Veterans Affairs.

As healthcare costs rise, physicians and other stakeholders are now seeking innovative and effective ways to reduce the provision of low-value services.^1,2 The Choosing Wisely^® campaign aims to further this goal by promoting lists of specific procedures, tests, and treatments that providers should avoid in selected clinical settings.³ On February 21, 2013, the Society of Hospital Medicine (SHM) released 2 Choosing Wisely^® lists consisting of adult and pediatric services that are seen as costly to consumers and to the healthcare system, but which are often nonbeneficial or even harmful.^4,5 A total of 80 physician and nurse specialty societies have joined in submitting additional lists.

Despite the growing enthusiasm for this effort, questions remain regarding the Choosing Wisely^® campaign’s ability to initiate the meaningful de-adoption of low-value services. Specifically, prior efforts to reduce the use of services deemed to be of questionable benefit have met several challenges.^2,6 Early analyses of the Choosing Wisely^® recommendations reveal similar roadblocks and variable uptakes of several recommendations.^7-10 While the reasons for difficulties in achieving de-adoption are broad, one important factor in whether clinicians are willing to follow guideline recommendations from such initiatives as Choosing Wisely^®is the extent to which they believe in the underlying evidence.¹¹ The current work seeks to formally evaluate the evidence supporting the Choosing Wisely^® recommendations, and to compare the quality of evidence supporting SHM lists to other published Choosing Wisely^® lists.

Data Sources

Using the online listing of published Choosing Wisely^® recommendations, a dataset was generated incorporating all 320 recommendations comprising the 58 lists published through August, 2014; these include both the adult and pediatric hospital medicine lists released by the SHM.^4,5,12 Although data collection ended at this point, this represents a majority of all 81 lists and 535 recommendations published through December, 2017. The reviewers (A.J.A., A.G., M.W., T.S.V., M.S., and C.R.C) extracted information about the references cited for each recommendation.

Data Analysis

The reviewers obtained each reference cited by a Choosing Wisely^® recommendation and categorized it by evidence strength along the following hierarchy: clinical practice guideline (CPG), primary research, review article, expert opinion, book, or others/unknown. CPGs were used as the highest level of evidence based on standard expectations for methodological rigor.¹³ Primary research was further rated as follows: systematic reviews and meta-analyses, randomized controlled trials (RCTs), observational studies, and case series. Each recommendation was graded using only the strongest piece of evidence cited.

Guideline Appraisal

We further sought to evaluate the strength of referenced CPGs. To accomplish this, a 10% random sample of the Choosing Wisely^® recommendations citing CPGs was selected, and the referenced CPGs were obtained. Separately, CPGs referenced by the SHM-published adult and pediatric lists were also obtained. For both groups, one CPG was randomly selected when a recommendation cited more than one CPG. These guidelines were assessed using the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument, a widely used instrument designed to assess CPG quality.^14,15 AGREE II consists of 25 questions categorized into 6 domains: scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence. Guidelines are also assigned an overall score. Two trained reviewers (A.J.A. and A.G.) assessed each of the sampled CPGs using a standardized form. Scores were then standardized using the method recommended by the instrument and reported as a percentage of available points. Although a standard interpretation of scores is not provided by the instrument, prior applications deemed scores below 50% as deficient^16,17. When a recommendation item cited multiple CPGs, one was randomly selected. We also abstracted data on the year of publication, the evidence grade assigned to specific items recommended by Choosing Wisely^®, and whether the CPG addressed the referring recommendation. All data management and analysis were conducted using Stata (V14.2, StataCorp, College Station, Texas).

A total of 320 recommendations were considered in our analysis, including 10 published across the 2 hospital medicine lists. When limited to the highest quality citation for each of the recommendations, 225 (70.3%) cited CPGs, whereas 71 (22.2%) cited primary research articles (Table 1). Specifically, 29 (9.1%) cited systematic reviews and meta-analyses, 28 (8.8%) cited observational studies, and 13 (4.1%) cited RCTs. One recommendation (0.3%) cited a case series as its highest level of evidence, 7 (2.2%) cited review articles, 7 (2.2%) cited editorials or opinion pieces, and 10 (3.1%) cited other types of documents, such as websites or books. Among hospital medicine recommendations, 9 (90%) referenced CPGs and 1 (10%) cited an observational study.

For the AGREE II assessment, we included 23 CPGs from the 225 referenced across all recommendations, after which we separately selected 6 CPGs from the hospital medicine recommendations. There was no overlap. Notably, 4 hospital medicine recommendations referenced a common CPG. Among the random sample of referenced CPGs, the median overall score obtained by using AGREE II was 54.2% (IQR 33.3%-70.8%, Table 2). This was similar to the median overall among hospital medicine guidelines (58.2%, IQR 50.0%-83.3%). Both hospital medicine and other sampled guidelines tended to score poorly in stakeholder involvement (48.6%, IQR 44.1%-61.1% and 47.2%, IQR 38.9%-61.1%, respectively). There were no significant differences between hospital medicine-referenced CPGs and the larger sample of CPGs in any AGREE II subdomains. The median age from the CPG publication to the list publication was 7 years (IQR 4–7) for hospital medicine recommendations and 3 years (IQR 2–6) for the nonhospital medicine recommendations. Substantial agreement was found between raters on the overall guideline assessment (ICC 0.80, 95% CI 0.58-0.91; Supplementary Table 1).

Given the rising costs and the potential for iatrogenic harm, curbing ineffective practices has become an urgent concern. To achieve this, the Choosing Wisely^® campaign has taken an important step by targeting certain low-value practices for de-adoption. However, the evidence supporting recommendations is variable. Specifically, 25 recommendations cited case series, review articles, or lower quality evidence as their highest level of support; moreover, among recommendations citing CPGs, quality, timeliness, and support for the recommendation item were variable. Although the hospital medicine lists tended to cite higher-quality evidence in the form of CPGs, these CPGs were often less recent than the guidelines referenced by other lists.

Our findings parallel those of other works that evaluate evidence among Choosing Wisely^® recommendations and, more broadly, among CPGs.^18–21 Lin and Yancey evaluated the quality of primary care-focused Choosing Wisely^® recommendations using the Strength of Recommendation Taxonomy, a ranking system that evaluates evidence quality, consistency, and patient-centeredness.¹⁸ In their analysis, the authors found that many recommendations were based on lower quality evidence or relied on nonpatent-centered intermediate outcomes. Several groups, meanwhile, have evaluated the quality of evidence supporting CPG recommendations, finding them to be highly variable as well.^19–21 These findings likely reflect inherent difficulties in the process, by which guideline development groups distill a broad evidence base into useful clinical recommendations, a reality that may have influenced the Choosing Wisely^® list development groups seeking to make similar recommendations on low-value services.

These data should be taken in context due to several limitations. First, our sample of referenced CPGs includes only a small sample of all CPGs cited; thus, it may not be representative of all referenced guidelines. Second, the AGREE II assessment is inherently subjective, despite the availability of training materials. Third, data collection ended in April, 2014. Although this represents a majority of published lists to date, it is possible that more recent Choosing Wisely^®lists include a stronger focus on evidence quality. Finally, references cited by Choosing Wisely^®may not be representative of the entirety of the dataset that was considered when formulating the recommendations.

Despite these limitations, our findings suggest that Choosing Wisely^®recommendations vary in terms of evidence strength. Although our results reveal that the majority of recommendations cite guidelines or high-quality original research, evidence gaps remain, with a small number citing low-quality evidence or low-quality CPGs as their highest form of support. Given the barriers to the successful de-implementation of low-value services, such campaigns as Choosing Wisely^®face an uphill battle in their attempt to prompt behavior changes among providers and consumers.^6-9 As a result, it is incumbent on funding agencies and medical journals to promote studies evaluating the harms and overall value of the care we deliver.

Although a majority of Choosing Wisely^® recommendations cite high-quality evidence, some reference low-quality evidence or low-quality CPGs as their highest form of support. To overcome clinical inertia and other barriers to the successful de-implementation of low-value services, a clear rationale for the impetus to eradicate entrenched practices is critical.^2,22 Choosing Wisely^® has provided visionary leadership and a powerful platform to question low-value care. To expand the campaign’s efforts, the medical field must be able to generate the high-quality evidence necessary to support these efforts; further, list development groups must consider the availability of strong evidence when targeting services for de-implementation.

This work was supported, in part, by a grant from the Agency for Healthcare Research and Quality (No. K08HS020672, Dr. Cooke).

Disclosures

The authors have nothing to disclose.

As healthcare costs rise, physicians and other stakeholders are now seeking innovative and effective ways to reduce the provision of low-value services.^1,2 The Choosing Wisely^® campaign aims to further this goal by promoting lists of specific procedures, tests, and treatments that providers should avoid in selected clinical settings.³ On February 21, 2013, the Society of Hospital Medicine (SHM) released 2 Choosing Wisely^® lists consisting of adult and pediatric services that are seen as costly to consumers and to the healthcare system, but which are often nonbeneficial or even harmful.^4,5 A total of 80 physician and nurse specialty societies have joined in submitting additional lists.

Despite the growing enthusiasm for this effort, questions remain regarding the Choosing Wisely^® campaign’s ability to initiate the meaningful de-adoption of low-value services. Specifically, prior efforts to reduce the use of services deemed to be of questionable benefit have met several challenges.^2,6 Early analyses of the Choosing Wisely^® recommendations reveal similar roadblocks and variable uptakes of several recommendations.^7-10 While the reasons for difficulties in achieving de-adoption are broad, one important factor in whether clinicians are willing to follow guideline recommendations from such initiatives as Choosing Wisely^®is the extent to which they believe in the underlying evidence.¹¹ The current work seeks to formally evaluate the evidence supporting the Choosing Wisely^® recommendations, and to compare the quality of evidence supporting SHM lists to other published Choosing Wisely^® lists.

Data Sources

Using the online listing of published Choosing Wisely^® recommendations, a dataset was generated incorporating all 320 recommendations comprising the 58 lists published through August, 2014; these include both the adult and pediatric hospital medicine lists released by the SHM.^4,5,12 Although data collection ended at this point, this represents a majority of all 81 lists and 535 recommendations published through December, 2017. The reviewers (A.J.A., A.G., M.W., T.S.V., M.S., and C.R.C) extracted information about the references cited for each recommendation.

Data Analysis

The reviewers obtained each reference cited by a Choosing Wisely^® recommendation and categorized it by evidence strength along the following hierarchy: clinical practice guideline (CPG), primary research, review article, expert opinion, book, or others/unknown. CPGs were used as the highest level of evidence based on standard expectations for methodological rigor.¹³ Primary research was further rated as follows: systematic reviews and meta-analyses, randomized controlled trials (RCTs), observational studies, and case series. Each recommendation was graded using only the strongest piece of evidence cited.

Guideline Appraisal

We further sought to evaluate the strength of referenced CPGs. To accomplish this, a 10% random sample of the Choosing Wisely^® recommendations citing CPGs was selected, and the referenced CPGs were obtained. Separately, CPGs referenced by the SHM-published adult and pediatric lists were also obtained. For both groups, one CPG was randomly selected when a recommendation cited more than one CPG. These guidelines were assessed using the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument, a widely used instrument designed to assess CPG quality.^14,15 AGREE II consists of 25 questions categorized into 6 domains: scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence. Guidelines are also assigned an overall score. Two trained reviewers (A.J.A. and A.G.) assessed each of the sampled CPGs using a standardized form. Scores were then standardized using the method recommended by the instrument and reported as a percentage of available points. Although a standard interpretation of scores is not provided by the instrument, prior applications deemed scores below 50% as deficient^16,17. When a recommendation item cited multiple CPGs, one was randomly selected. We also abstracted data on the year of publication, the evidence grade assigned to specific items recommended by Choosing Wisely^®, and whether the CPG addressed the referring recommendation. All data management and analysis were conducted using Stata (V14.2, StataCorp, College Station, Texas).

A total of 320 recommendations were considered in our analysis, including 10 published across the 2 hospital medicine lists. When limited to the highest quality citation for each of the recommendations, 225 (70.3%) cited CPGs, whereas 71 (22.2%) cited primary research articles (Table 1). Specifically, 29 (9.1%) cited systematic reviews and meta-analyses, 28 (8.8%) cited observational studies, and 13 (4.1%) cited RCTs. One recommendation (0.3%) cited a case series as its highest level of evidence, 7 (2.2%) cited review articles, 7 (2.2%) cited editorials or opinion pieces, and 10 (3.1%) cited other types of documents, such as websites or books. Among hospital medicine recommendations, 9 (90%) referenced CPGs and 1 (10%) cited an observational study.

For the AGREE II assessment, we included 23 CPGs from the 225 referenced across all recommendations, after which we separately selected 6 CPGs from the hospital medicine recommendations. There was no overlap. Notably, 4 hospital medicine recommendations referenced a common CPG. Among the random sample of referenced CPGs, the median overall score obtained by using AGREE II was 54.2% (IQR 33.3%-70.8%, Table 2). This was similar to the median overall among hospital medicine guidelines (58.2%, IQR 50.0%-83.3%). Both hospital medicine and other sampled guidelines tended to score poorly in stakeholder involvement (48.6%, IQR 44.1%-61.1% and 47.2%, IQR 38.9%-61.1%, respectively). There were no significant differences between hospital medicine-referenced CPGs and the larger sample of CPGs in any AGREE II subdomains. The median age from the CPG publication to the list publication was 7 years (IQR 4–7) for hospital medicine recommendations and 3 years (IQR 2–6) for the nonhospital medicine recommendations. Substantial agreement was found between raters on the overall guideline assessment (ICC 0.80, 95% CI 0.58-0.91; Supplementary Table 1).

Given the rising costs and the potential for iatrogenic harm, curbing ineffective practices has become an urgent concern. To achieve this, the Choosing Wisely^® campaign has taken an important step by targeting certain low-value practices for de-adoption. However, the evidence supporting recommendations is variable. Specifically, 25 recommendations cited case series, review articles, or lower quality evidence as their highest level of support; moreover, among recommendations citing CPGs, quality, timeliness, and support for the recommendation item were variable. Although the hospital medicine lists tended to cite higher-quality evidence in the form of CPGs, these CPGs were often less recent than the guidelines referenced by other lists.

Our findings parallel those of other works that evaluate evidence among Choosing Wisely^® recommendations and, more broadly, among CPGs.^18–21 Lin and Yancey evaluated the quality of primary care-focused Choosing Wisely^® recommendations using the Strength of Recommendation Taxonomy, a ranking system that evaluates evidence quality, consistency, and patient-centeredness.¹⁸ In their analysis, the authors found that many recommendations were based on lower quality evidence or relied on nonpatent-centered intermediate outcomes. Several groups, meanwhile, have evaluated the quality of evidence supporting CPG recommendations, finding them to be highly variable as well.^19–21 These findings likely reflect inherent difficulties in the process, by which guideline development groups distill a broad evidence base into useful clinical recommendations, a reality that may have influenced the Choosing Wisely^® list development groups seeking to make similar recommendations on low-value services.

These data should be taken in context due to several limitations. First, our sample of referenced CPGs includes only a small sample of all CPGs cited; thus, it may not be representative of all referenced guidelines. Second, the AGREE II assessment is inherently subjective, despite the availability of training materials. Third, data collection ended in April, 2014. Although this represents a majority of published lists to date, it is possible that more recent Choosing Wisely^®lists include a stronger focus on evidence quality. Finally, references cited by Choosing Wisely^®may not be representative of the entirety of the dataset that was considered when formulating the recommendations.

Despite these limitations, our findings suggest that Choosing Wisely^®recommendations vary in terms of evidence strength. Although our results reveal that the majority of recommendations cite guidelines or high-quality original research, evidence gaps remain, with a small number citing low-quality evidence or low-quality CPGs as their highest form of support. Given the barriers to the successful de-implementation of low-value services, such campaigns as Choosing Wisely^®face an uphill battle in their attempt to prompt behavior changes among providers and consumers.^6-9 As a result, it is incumbent on funding agencies and medical journals to promote studies evaluating the harms and overall value of the care we deliver.

Although a majority of Choosing Wisely^® recommendations cite high-quality evidence, some reference low-quality evidence or low-quality CPGs as their highest form of support. To overcome clinical inertia and other barriers to the successful de-implementation of low-value services, a clear rationale for the impetus to eradicate entrenched practices is critical.^2,22 Choosing Wisely^® has provided visionary leadership and a powerful platform to question low-value care. To expand the campaign’s efforts, the medical field must be able to generate the high-quality evidence necessary to support these efforts; further, list development groups must consider the availability of strong evidence when targeting services for de-implementation.

This work was supported, in part, by a grant from the Agency for Healthcare Research and Quality (No. K08HS020672, Dr. Cooke).

Disclosures

The authors have nothing to disclose.

As healthcare costs rise, physicians and other stakeholders are now seeking innovative and effective ways to reduce the provision of low-value services.^1,2 The Choosing Wisely^® campaign aims to further this goal by promoting lists of specific procedures, tests, and treatments that providers should avoid in selected clinical settings.³ On February 21, 2013, the Society of Hospital Medicine (SHM) released 2 Choosing Wisely^® lists consisting of adult and pediatric services that are seen as costly to consumers and to the healthcare system, but which are often nonbeneficial or even harmful.^4,5 A total of 80 physician and nurse specialty societies have joined in submitting additional lists.

Despite the growing enthusiasm for this effort, questions remain regarding the Choosing Wisely^® campaign’s ability to initiate the meaningful de-adoption of low-value services. Specifically, prior efforts to reduce the use of services deemed to be of questionable benefit have met several challenges.^2,6 Early analyses of the Choosing Wisely^® recommendations reveal similar roadblocks and variable uptakes of several recommendations.^7-10 While the reasons for difficulties in achieving de-adoption are broad, one important factor in whether clinicians are willing to follow guideline recommendations from such initiatives as Choosing Wisely^®is the extent to which they believe in the underlying evidence.¹¹ The current work seeks to formally evaluate the evidence supporting the Choosing Wisely^® recommendations, and to compare the quality of evidence supporting SHM lists to other published Choosing Wisely^® lists.

Data Sources

Using the online listing of published Choosing Wisely^® recommendations, a dataset was generated incorporating all 320 recommendations comprising the 58 lists published through August, 2014; these include both the adult and pediatric hospital medicine lists released by the SHM.^4,5,12 Although data collection ended at this point, this represents a majority of all 81 lists and 535 recommendations published through December, 2017. The reviewers (A.J.A., A.G., M.W., T.S.V., M.S., and C.R.C) extracted information about the references cited for each recommendation.

Data Analysis

The reviewers obtained each reference cited by a Choosing Wisely^® recommendation and categorized it by evidence strength along the following hierarchy: clinical practice guideline (CPG), primary research, review article, expert opinion, book, or others/unknown. CPGs were used as the highest level of evidence based on standard expectations for methodological rigor.¹³ Primary research was further rated as follows: systematic reviews and meta-analyses, randomized controlled trials (RCTs), observational studies, and case series. Each recommendation was graded using only the strongest piece of evidence cited.

Guideline Appraisal

We further sought to evaluate the strength of referenced CPGs. To accomplish this, a 10% random sample of the Choosing Wisely^® recommendations citing CPGs was selected, and the referenced CPGs were obtained. Separately, CPGs referenced by the SHM-published adult and pediatric lists were also obtained. For both groups, one CPG was randomly selected when a recommendation cited more than one CPG. These guidelines were assessed using the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument, a widely used instrument designed to assess CPG quality.^14,15 AGREE II consists of 25 questions categorized into 6 domains: scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence. Guidelines are also assigned an overall score. Two trained reviewers (A.J.A. and A.G.) assessed each of the sampled CPGs using a standardized form. Scores were then standardized using the method recommended by the instrument and reported as a percentage of available points. Although a standard interpretation of scores is not provided by the instrument, prior applications deemed scores below 50% as deficient^16,17. When a recommendation item cited multiple CPGs, one was randomly selected. We also abstracted data on the year of publication, the evidence grade assigned to specific items recommended by Choosing Wisely^®, and whether the CPG addressed the referring recommendation. All data management and analysis were conducted using Stata (V14.2, StataCorp, College Station, Texas).

A total of 320 recommendations were considered in our analysis, including 10 published across the 2 hospital medicine lists. When limited to the highest quality citation for each of the recommendations, 225 (70.3%) cited CPGs, whereas 71 (22.2%) cited primary research articles (Table 1). Specifically, 29 (9.1%) cited systematic reviews and meta-analyses, 28 (8.8%) cited observational studies, and 13 (4.1%) cited RCTs. One recommendation (0.3%) cited a case series as its highest level of evidence, 7 (2.2%) cited review articles, 7 (2.2%) cited editorials or opinion pieces, and 10 (3.1%) cited other types of documents, such as websites or books. Among hospital medicine recommendations, 9 (90%) referenced CPGs and 1 (10%) cited an observational study.

For the AGREE II assessment, we included 23 CPGs from the 225 referenced across all recommendations, after which we separately selected 6 CPGs from the hospital medicine recommendations. There was no overlap. Notably, 4 hospital medicine recommendations referenced a common CPG. Among the random sample of referenced CPGs, the median overall score obtained by using AGREE II was 54.2% (IQR 33.3%-70.8%, Table 2). This was similar to the median overall among hospital medicine guidelines (58.2%, IQR 50.0%-83.3%). Both hospital medicine and other sampled guidelines tended to score poorly in stakeholder involvement (48.6%, IQR 44.1%-61.1% and 47.2%, IQR 38.9%-61.1%, respectively). There were no significant differences between hospital medicine-referenced CPGs and the larger sample of CPGs in any AGREE II subdomains. The median age from the CPG publication to the list publication was 7 years (IQR 4–7) for hospital medicine recommendations and 3 years (IQR 2–6) for the nonhospital medicine recommendations. Substantial agreement was found between raters on the overall guideline assessment (ICC 0.80, 95% CI 0.58-0.91; Supplementary Table 1).

Given the rising costs and the potential for iatrogenic harm, curbing ineffective practices has become an urgent concern. To achieve this, the Choosing Wisely^® campaign has taken an important step by targeting certain low-value practices for de-adoption. However, the evidence supporting recommendations is variable. Specifically, 25 recommendations cited case series, review articles, or lower quality evidence as their highest level of support; moreover, among recommendations citing CPGs, quality, timeliness, and support for the recommendation item were variable. Although the hospital medicine lists tended to cite higher-quality evidence in the form of CPGs, these CPGs were often less recent than the guidelines referenced by other lists.

Our findings parallel those of other works that evaluate evidence among Choosing Wisely^® recommendations and, more broadly, among CPGs.^18–21 Lin and Yancey evaluated the quality of primary care-focused Choosing Wisely^® recommendations using the Strength of Recommendation Taxonomy, a ranking system that evaluates evidence quality, consistency, and patient-centeredness.¹⁸ In their analysis, the authors found that many recommendations were based on lower quality evidence or relied on nonpatent-centered intermediate outcomes. Several groups, meanwhile, have evaluated the quality of evidence supporting CPG recommendations, finding them to be highly variable as well.^19–21 These findings likely reflect inherent difficulties in the process, by which guideline development groups distill a broad evidence base into useful clinical recommendations, a reality that may have influenced the Choosing Wisely^® list development groups seeking to make similar recommendations on low-value services.

These data should be taken in context due to several limitations. First, our sample of referenced CPGs includes only a small sample of all CPGs cited; thus, it may not be representative of all referenced guidelines. Second, the AGREE II assessment is inherently subjective, despite the availability of training materials. Third, data collection ended in April, 2014. Although this represents a majority of published lists to date, it is possible that more recent Choosing Wisely^®lists include a stronger focus on evidence quality. Finally, references cited by Choosing Wisely^®may not be representative of the entirety of the dataset that was considered when formulating the recommendations.

Despite these limitations, our findings suggest that Choosing Wisely^®recommendations vary in terms of evidence strength. Although our results reveal that the majority of recommendations cite guidelines or high-quality original research, evidence gaps remain, with a small number citing low-quality evidence or low-quality CPGs as their highest form of support. Given the barriers to the successful de-implementation of low-value services, such campaigns as Choosing Wisely^®face an uphill battle in their attempt to prompt behavior changes among providers and consumers.^6-9 As a result, it is incumbent on funding agencies and medical journals to promote studies evaluating the harms and overall value of the care we deliver.

Although a majority of Choosing Wisely^® recommendations cite high-quality evidence, some reference low-quality evidence or low-quality CPGs as their highest form of support. To overcome clinical inertia and other barriers to the successful de-implementation of low-value services, a clear rationale for the impetus to eradicate entrenched practices is critical.^2,22 Choosing Wisely^® has provided visionary leadership and a powerful platform to question low-value care. To expand the campaign’s efforts, the medical field must be able to generate the high-quality evidence necessary to support these efforts; further, list development groups must consider the availability of strong evidence when targeting services for de-implementation.

This work was supported, in part, by a grant from the Agency for Healthcare Research and Quality (No. K08HS020672, Dr. Cooke).

Disclosures

The authors have nothing to disclose.

Recent years have seen a proliferation of programs designed to prevent readmissions, including patient education initiatives, financial assistance programs, postdischarge services, and clinical personnel assigned to help patients navigate their posthospitalization clinical care. Although some strategies do not require direct patient participation (such as timely and effective handoffs between inpatient and outpatient care teams), many rely upon a commitment by the patient to participate in the postdischarge care plan. At our hospital, we have found that only about 2/3 of patients who are offered transitional interventions (such as postdischarge phone calls by nurses or home nursing through a “transition guide” program) receive the intended interventions, and those who do not receive them are more likely to be readmitted.¹ While limited patient uptake may relate, in part, to factors that are difficult to overcome, such as inadequate housing or phone service, we have also encountered patients whose values, beliefs, or preferences about their care do not align with those of the care team. The purposes of this exploratory study were to (1) assess patient attitudes surrounding readmission, (2) ascertain whether these attitudes are associated with actual readmission, and (3) determine whether patients can estimate their own risk of readmission.

From January 2014 to September 2016, we circulated surveys to patients on internal medicine nursing units who were being discharged home within 24 hours. Blank surveys were distributed to nursing units by the researchers. Unit clerks and support staff were educated on the purpose of the project and asked to distribute surveys to patients who were identified by unit case managers or nurses as slated for discharge. Staff members were not asked to help with or supervise survey completion. Surveys were generally filled out by patients, but we allowed family members to assist patients if needed, and to indicate so with a checkbox. There were no exclusion criteria. Because surveys were distributed by clinical staff, the received surveys can be considered a convenience sample. Patients were asked 5 questions with 4- or 5-point Likert scale responses:

(1) “How likely is it that you will be admitted to the hospital (have to stay in the hospital overnight) again within the next 30 days after you leave the hospital this time?” [answers ranging from “Very Unlikely (<5% chance)” to “Very Likely (>50% chance)”];

(2) “How would you feel about being rehospitalized in the next month?” [answers ranging from “Very sad, frustrated, or disappointed” to “Very happy or relieved”];

(3) “How much do you think that you personally can control whether or not you will be rehospitalized (based on what you do to take care of your body, take your medicines, and follow-up with your healthcare team)?” [answers ranging from “I have no control over whether I will be rehospitalized” to “I have complete control over whether I will be rehospitalized”];

(4) “Which of the options below best describes how you plan to follow the medical instructions after you leave the hospital?” [answers ranging from “I do NOT plan to do very much of what I am being asked to do by the doctors, nurses, therapists, and other members of the care team” to “I plan to do EVERYTHING I am being asked to do by the doctors, nurses, therapists and other members of the care team”]; and

(5) “Pick the item below that best describes YOUR OWN VIEW of the care team’s recommendations:” [answers ranging from “I DO NOT AGREE AT ALL that the best way to be healthy is to do exactly what I am being asked to do by the doctors, nurses, therapists, and other members of the care team” to “I FULLY AGREE that the best way to be healthy is to do exactly what I am being asked to do by the doctors, nurses, therapists, and other members of the care team”].

Responses were linked, based on discharge date and medical record number, to administrative data, including age, sex, race, payer, and clinical data. Subsequent hospitalizations to our hospital were ascertained from administrative data. We estimated expected risk of readmission using the all payer refined diagnosis related group coupled with the associated severity-of-illness (SOI) score, as we have reported previously.^2-5 We restricted our analysis to patients who answered the question related to the likelihood of readmission. Logistic regression models were constructed using actual 30-day readmission as the dependent variable to determine whether patients could predict their own readmissions and whether patient attitudes and beliefs about their care were predictive of subsequent readmission. Patient survey responses were entered as continuous independent variables (ranging from 1-4 or 1-5, as appropriate). Multivariable logistic regression was used to determine whether patients could predict their readmissions independent of demographic variables and expected readmission rate (modeled continuously); we repeated this model after dichotomizing the patient’s estimate of the likelihood of readmission as either “unlikely” or “likely.” Patients with missing survey responses were excluded from individual models without imputation. The study was approved by the Johns Hopkins institutional review board.

Responses were obtained from 895 patients. Their median age was 56 years [interquartile range, 43-67], 51.4% were female, and 41.7% were white. Mean SOI was 2.53 (on a 1-4 scale), and median length-of-stay was representative for our medical service at 5.2 days (range, 1-66 days). Family members reported filling out the survey in 57 cases. The primary payer was Medicare in 40.7%, Medicaid in 24.9%, and other in 34.4%. A total of 138 patients (15.4%) were readmitted within 30 days. The Table shows survey responses and associated readmission rates. None of the attitudes related to readmission were predictive of actual readmission. However, patients were able to predict their own readmissions (P = .002 for linear trend). After adjustment for expected readmission rate, race, sex, age, and payer, the trend remained significant (P = .005). Other significant predictors of readmissions in this model included expected readmission rate (P = .002), age (P = .02), and payer (P = .002). After dichotomizing the patient estimate of readmission rate as “unlikely” (N = 581) or “likely” (N = 314), the unadjusted odds ratio associating a patient-estimated risk of readmission as “likely” with actual readmission was 1.8 (95% confidence interval, 1.2-2.5). The adjusted odds ratio (including the variables above) was 1.6 (1.1-2.4).

Our findings demonstrate that patients are able to quantify their own readmission risk. This was true even after adjustment for expected readmission rate, age, sex, race, and payer. However, we did not identify any patient attitudes, beliefs, or preferences related to readmission or discharge instructions that were associated with subsequent rehospitalization. Reassuringly, more than 80% of patients who responded to the survey indicated that they would be sad, frustrated, or disappointed should readmission occur. This suggests that most patients are invested in preventing rehospitalization. Also reassuring was that patients indicated that they agreed with the discharge care plan and intended to follow their discharge instructions.

The major limitation of this study is that it was a convenience sample. Surveys were distributed inconsistently by nursing unit staff, preventing us from calculating a response rate. Further, it is possible, if not likely, that those patients with higher levels of engagement were more likely to take the time to respond, enriching our sample with activated patients. Although we allowed family members to fill out surveys on behalf of patients, this was done in fewer than 10% of instances; as such, our data may have limited applicability to patients who are physically or cognitively unable to participate in the discharge process. Finally, in this study, we did not capture readmissions to other facilities.

We conclude that patients are able to predict their own readmissions, even after accounting for other potential predictors of readmission. However, we found no evidence to support the possibility that low levels of engagement, limited trust in the healthcare team, or nonchalance about being readmitted are associated with subsequent rehospitalization. Whether asking patients about their perceived risk of readmission might help target readmission prevention programs deserves further study.

Dr. Daniel J. Brotman had full access to the data in the study and takes responsibility for the integrity of the study data and the accuracy of the data analysis. The authors also thank the following individuals for their contributions: Drafting the manuscript (Brotman); revising the manuscript for important intellectual content (Brotman, Shihab, Tieu, Cheng, Bertram, Hoyer, Deutschendorf); acquiring the data (Brotman, Shihab, Tieu, Cheng, Bertram, Deutschendorf); interpreting the data (Brotman, Shihab, Tieu, Cheng, Bertram, Hoyer, Deutschendorf); and analyzing the data (Brotman). The authors thank nursing leadership and nursing unit staff for their assistance in distributing surveys.

Funding support: Johns Hopkins Hospitalist Scholars Program

Disclosures: The authors have declared no conflicts of interest.

Recent years have seen a proliferation of programs designed to prevent readmissions, including patient education initiatives, financial assistance programs, postdischarge services, and clinical personnel assigned to help patients navigate their posthospitalization clinical care. Although some strategies do not require direct patient participation (such as timely and effective handoffs between inpatient and outpatient care teams), many rely upon a commitment by the patient to participate in the postdischarge care plan. At our hospital, we have found that only about 2/3 of patients who are offered transitional interventions (such as postdischarge phone calls by nurses or home nursing through a “transition guide” program) receive the intended interventions, and those who do not receive them are more likely to be readmitted.¹ While limited patient uptake may relate, in part, to factors that are difficult to overcome, such as inadequate housing or phone service, we have also encountered patients whose values, beliefs, or preferences about their care do not align with those of the care team. The purposes of this exploratory study were to (1) assess patient attitudes surrounding readmission, (2) ascertain whether these attitudes are associated with actual readmission, and (3) determine whether patients can estimate their own risk of readmission.

From January 2014 to September 2016, we circulated surveys to patients on internal medicine nursing units who were being discharged home within 24 hours. Blank surveys were distributed to nursing units by the researchers. Unit clerks and support staff were educated on the purpose of the project and asked to distribute surveys to patients who were identified by unit case managers or nurses as slated for discharge. Staff members were not asked to help with or supervise survey completion. Surveys were generally filled out by patients, but we allowed family members to assist patients if needed, and to indicate so with a checkbox. There were no exclusion criteria. Because surveys were distributed by clinical staff, the received surveys can be considered a convenience sample. Patients were asked 5 questions with 4- or 5-point Likert scale responses:

(1) “How likely is it that you will be admitted to the hospital (have to stay in the hospital overnight) again within the next 30 days after you leave the hospital this time?” [answers ranging from “Very Unlikely (<5% chance)” to “Very Likely (>50% chance)”];

(2) “How would you feel about being rehospitalized in the next month?” [answers ranging from “Very sad, frustrated, or disappointed” to “Very happy or relieved”];

(3) “How much do you think that you personally can control whether or not you will be rehospitalized (based on what you do to take care of your body, take your medicines, and follow-up with your healthcare team)?” [answers ranging from “I have no control over whether I will be rehospitalized” to “I have complete control over whether I will be rehospitalized”];

(4) “Which of the options below best describes how you plan to follow the medical instructions after you leave the hospital?” [answers ranging from “I do NOT plan to do very much of what I am being asked to do by the doctors, nurses, therapists, and other members of the care team” to “I plan to do EVERYTHING I am being asked to do by the doctors, nurses, therapists and other members of the care team”]; and

(5) “Pick the item below that best describes YOUR OWN VIEW of the care team’s recommendations:” [answers ranging from “I DO NOT AGREE AT ALL that the best way to be healthy is to do exactly what I am being asked to do by the doctors, nurses, therapists, and other members of the care team” to “I FULLY AGREE that the best way to be healthy is to do exactly what I am being asked to do by the doctors, nurses, therapists, and other members of the care team”].

Responses were linked, based on discharge date and medical record number, to administrative data, including age, sex, race, payer, and clinical data. Subsequent hospitalizations to our hospital were ascertained from administrative data. We estimated expected risk of readmission using the all payer refined diagnosis related group coupled with the associated severity-of-illness (SOI) score, as we have reported previously.^2-5 We restricted our analysis to patients who answered the question related to the likelihood of readmission. Logistic regression models were constructed using actual 30-day readmission as the dependent variable to determine whether patients could predict their own readmissions and whether patient attitudes and beliefs about their care were predictive of subsequent readmission. Patient survey responses were entered as continuous independent variables (ranging from 1-4 or 1-5, as appropriate). Multivariable logistic regression was used to determine whether patients could predict their readmissions independent of demographic variables and expected readmission rate (modeled continuously); we repeated this model after dichotomizing the patient’s estimate of the likelihood of readmission as either “unlikely” or “likely.” Patients with missing survey responses were excluded from individual models without imputation. The study was approved by the Johns Hopkins institutional review board.

Responses were obtained from 895 patients. Their median age was 56 years [interquartile range, 43-67], 51.4% were female, and 41.7% were white. Mean SOI was 2.53 (on a 1-4 scale), and median length-of-stay was representative for our medical service at 5.2 days (range, 1-66 days). Family members reported filling out the survey in 57 cases. The primary payer was Medicare in 40.7%, Medicaid in 24.9%, and other in 34.4%. A total of 138 patients (15.4%) were readmitted within 30 days. The Table shows survey responses and associated readmission rates. None of the attitudes related to readmission were predictive of actual readmission. However, patients were able to predict their own readmissions (P = .002 for linear trend). After adjustment for expected readmission rate, race, sex, age, and payer, the trend remained significant (P = .005). Other significant predictors of readmissions in this model included expected readmission rate (P = .002), age (P = .02), and payer (P = .002). After dichotomizing the patient estimate of readmission rate as “unlikely” (N = 581) or “likely” (N = 314), the unadjusted odds ratio associating a patient-estimated risk of readmission as “likely” with actual readmission was 1.8 (95% confidence interval, 1.2-2.5). The adjusted odds ratio (including the variables above) was 1.6 (1.1-2.4).

Our findings demonstrate that patients are able to quantify their own readmission risk. This was true even after adjustment for expected readmission rate, age, sex, race, and payer. However, we did not identify any patient attitudes, beliefs, or preferences related to readmission or discharge instructions that were associated with subsequent rehospitalization. Reassuringly, more than 80% of patients who responded to the survey indicated that they would be sad, frustrated, or disappointed should readmission occur. This suggests that most patients are invested in preventing rehospitalization. Also reassuring was that patients indicated that they agreed with the discharge care plan and intended to follow their discharge instructions.

The major limitation of this study is that it was a convenience sample. Surveys were distributed inconsistently by nursing unit staff, preventing us from calculating a response rate. Further, it is possible, if not likely, that those patients with higher levels of engagement were more likely to take the time to respond, enriching our sample with activated patients. Although we allowed family members to fill out surveys on behalf of patients, this was done in fewer than 10% of instances; as such, our data may have limited applicability to patients who are physically or cognitively unable to participate in the discharge process. Finally, in this study, we did not capture readmissions to other facilities.

We conclude that patients are able to predict their own readmissions, even after accounting for other potential predictors of readmission. However, we found no evidence to support the possibility that low levels of engagement, limited trust in the healthcare team, or nonchalance about being readmitted are associated with subsequent rehospitalization. Whether asking patients about their perceived risk of readmission might help target readmission prevention programs deserves further study.

Dr. Daniel J. Brotman had full access to the data in the study and takes responsibility for the integrity of the study data and the accuracy of the data analysis. The authors also thank the following individuals for their contributions: Drafting the manuscript (Brotman); revising the manuscript for important intellectual content (Brotman, Shihab, Tieu, Cheng, Bertram, Hoyer, Deutschendorf); acquiring the data (Brotman, Shihab, Tieu, Cheng, Bertram, Deutschendorf); interpreting the data (Brotman, Shihab, Tieu, Cheng, Bertram, Hoyer, Deutschendorf); and analyzing the data (Brotman). The authors thank nursing leadership and nursing unit staff for their assistance in distributing surveys.

Funding support: Johns Hopkins Hospitalist Scholars Program

Disclosures: The authors have declared no conflicts of interest.

Recent years have seen a proliferation of programs designed to prevent readmissions, including patient education initiatives, financial assistance programs, postdischarge services, and clinical personnel assigned to help patients navigate their posthospitalization clinical care. Although some strategies do not require direct patient participation (such as timely and effective handoffs between inpatient and outpatient care teams), many rely upon a commitment by the patient to participate in the postdischarge care plan. At our hospital, we have found that only about 2/3 of patients who are offered transitional interventions (such as postdischarge phone calls by nurses or home nursing through a “transition guide” program) receive the intended interventions, and those who do not receive them are more likely to be readmitted.¹ While limited patient uptake may relate, in part, to factors that are difficult to overcome, such as inadequate housing or phone service, we have also encountered patients whose values, beliefs, or preferences about their care do not align with those of the care team. The purposes of this exploratory study were to (1) assess patient attitudes surrounding readmission, (2) ascertain whether these attitudes are associated with actual readmission, and (3) determine whether patients can estimate their own risk of readmission.

From January 2014 to September 2016, we circulated surveys to patients on internal medicine nursing units who were being discharged home within 24 hours. Blank surveys were distributed to nursing units by the researchers. Unit clerks and support staff were educated on the purpose of the project and asked to distribute surveys to patients who were identified by unit case managers or nurses as slated for discharge. Staff members were not asked to help with or supervise survey completion. Surveys were generally filled out by patients, but we allowed family members to assist patients if needed, and to indicate so with a checkbox. There were no exclusion criteria. Because surveys were distributed by clinical staff, the received surveys can be considered a convenience sample. Patients were asked 5 questions with 4- or 5-point Likert scale responses:

(1) “How likely is it that you will be admitted to the hospital (have to stay in the hospital overnight) again within the next 30 days after you leave the hospital this time?” [answers ranging from “Very Unlikely (<5% chance)” to “Very Likely (>50% chance)”];

(2) “How would you feel about being rehospitalized in the next month?” [answers ranging from “Very sad, frustrated, or disappointed” to “Very happy or relieved”];

(3) “How much do you think that you personally can control whether or not you will be rehospitalized (based on what you do to take care of your body, take your medicines, and follow-up with your healthcare team)?” [answers ranging from “I have no control over whether I will be rehospitalized” to “I have complete control over whether I will be rehospitalized”];

(4) “Which of the options below best describes how you plan to follow the medical instructions after you leave the hospital?” [answers ranging from “I do NOT plan to do very much of what I am being asked to do by the doctors, nurses, therapists, and other members of the care team” to “I plan to do EVERYTHING I am being asked to do by the doctors, nurses, therapists and other members of the care team”]; and

(5) “Pick the item below that best describes YOUR OWN VIEW of the care team’s recommendations:” [answers ranging from “I DO NOT AGREE AT ALL that the best way to be healthy is to do exactly what I am being asked to do by the doctors, nurses, therapists, and other members of the care team” to “I FULLY AGREE that the best way to be healthy is to do exactly what I am being asked to do by the doctors, nurses, therapists, and other members of the care team”].

Responses were linked, based on discharge date and medical record number, to administrative data, including age, sex, race, payer, and clinical data. Subsequent hospitalizations to our hospital were ascertained from administrative data. We estimated expected risk of readmission using the all payer refined diagnosis related group coupled with the associated severity-of-illness (SOI) score, as we have reported previously.^2-5 We restricted our analysis to patients who answered the question related to the likelihood of readmission. Logistic regression models were constructed using actual 30-day readmission as the dependent variable to determine whether patients could predict their own readmissions and whether patient attitudes and beliefs about their care were predictive of subsequent readmission. Patient survey responses were entered as continuous independent variables (ranging from 1-4 or 1-5, as appropriate). Multivariable logistic regression was used to determine whether patients could predict their readmissions independent of demographic variables and expected readmission rate (modeled continuously); we repeated this model after dichotomizing the patient’s estimate of the likelihood of readmission as either “unlikely” or “likely.” Patients with missing survey responses were excluded from individual models without imputation. The study was approved by the Johns Hopkins institutional review board.

Responses were obtained from 895 patients. Their median age was 56 years [interquartile range, 43-67], 51.4% were female, and 41.7% were white. Mean SOI was 2.53 (on a 1-4 scale), and median length-of-stay was representative for our medical service at 5.2 days (range, 1-66 days). Family members reported filling out the survey in 57 cases. The primary payer was Medicare in 40.7%, Medicaid in 24.9%, and other in 34.4%. A total of 138 patients (15.4%) were readmitted within 30 days. The Table shows survey responses and associated readmission rates. None of the attitudes related to readmission were predictive of actual readmission. However, patients were able to predict their own readmissions (P = .002 for linear trend). After adjustment for expected readmission rate, race, sex, age, and payer, the trend remained significant (P = .005). Other significant predictors of readmissions in this model included expected readmission rate (P = .002), age (P = .02), and payer (P = .002). After dichotomizing the patient estimate of readmission rate as “unlikely” (N = 581) or “likely” (N = 314), the unadjusted odds ratio associating a patient-estimated risk of readmission as “likely” with actual readmission was 1.8 (95% confidence interval, 1.2-2.5). The adjusted odds ratio (including the variables above) was 1.6 (1.1-2.4).

Our findings demonstrate that patients are able to quantify their own readmission risk. This was true even after adjustment for expected readmission rate, age, sex, race, and payer. However, we did not identify any patient attitudes, beliefs, or preferences related to readmission or discharge instructions that were associated with subsequent rehospitalization. Reassuringly, more than 80% of patients who responded to the survey indicated that they would be sad, frustrated, or disappointed should readmission occur. This suggests that most patients are invested in preventing rehospitalization. Also reassuring was that patients indicated that they agreed with the discharge care plan and intended to follow their discharge instructions.

The major limitation of this study is that it was a convenience sample. Surveys were distributed inconsistently by nursing unit staff, preventing us from calculating a response rate. Further, it is possible, if not likely, that those patients with higher levels of engagement were more likely to take the time to respond, enriching our sample with activated patients. Although we allowed family members to fill out surveys on behalf of patients, this was done in fewer than 10% of instances; as such, our data may have limited applicability to patients who are physically or cognitively unable to participate in the discharge process. Finally, in this study, we did not capture readmissions to other facilities.

We conclude that patients are able to predict their own readmissions, even after accounting for other potential predictors of readmission. However, we found no evidence to support the possibility that low levels of engagement, limited trust in the healthcare team, or nonchalance about being readmitted are associated with subsequent rehospitalization. Whether asking patients about their perceived risk of readmission might help target readmission prevention programs deserves further study.

Dr. Daniel J. Brotman had full access to the data in the study and takes responsibility for the integrity of the study data and the accuracy of the data analysis. The authors also thank the following individuals for their contributions: Drafting the manuscript (Brotman); revising the manuscript for important intellectual content (Brotman, Shihab, Tieu, Cheng, Bertram, Hoyer, Deutschendorf); acquiring the data (Brotman, Shihab, Tieu, Cheng, Bertram, Deutschendorf); interpreting the data (Brotman, Shihab, Tieu, Cheng, Bertram, Hoyer, Deutschendorf); and analyzing the data (Brotman). The authors thank nursing leadership and nursing unit staff for their assistance in distributing surveys.

Funding support: Johns Hopkins Hospitalist Scholars Program

Disclosures: The authors have declared no conflicts of interest.

In addition to treating patients, physicians frequently have other time commitments that could include administrative, teaching, research, and family duties. Inpatient medicine is particularly unforgiving to these nonclinical duties since patients have to be assessed on a daily basis. Because of this characteristic, it is not uncommon for inpatient care responsibility to be switched between physicians to create time for nonclinical duties and personal health.

In contrast to the ambulatory setting, the influence of physician continuity of care on inpatient outcomes has not been studied frequently. Studies of inpatient continuity have primarily focused on patient discharge (likely because of its objective nature) over the weekends (likely because weekend cross-coverage is common) and have reported conflicting results.^1-3 However, discontinuity of care is not isolated to the weekend since hospitalist-switches can occur at any time. In addition, expressing hospitalist continuity of care as a dichotomous variable (Was there weekend cross-coverage?) could incompletely express continuity since discharge likelihood might change with the consecutive number of days that a hospitalist is on service. This study measured the influence of hospitalist continuity throughout the patient’s hospitalization (rather than just the weekend) on daily patient discharge.

Study Setting and Databases Used for Analysis

The study was conducted at The Ottawa Hospital, Ontario, Canada, a 1000-bed teaching hospital with 2 campuses and the primary referral center in our region. The division of general internal medicine has 6 patient services (or “teams”) at two campuses led by a staff hospitalist (exclusively general internists), a senior medical resident (2nd year of training), and various numbers of interns and medical students. Staff hospitalists do not treat more than one patient service even on the weekends.

Patients are admitted to each service on a daily basis and almost exclusively from the emergency room. Assignment of patients is essentially random since all services have the same clinical expertise. At a particular campus, the number of patients assigned daily to each service is usually equivalent between teams. Patients almost never switch between teams but may be transferred to another specialty. The study was approved by our local research ethics board.

The Patient Registry Database records for each patient the date and time of admissions (defined as the moment that a patient’s admission request is entered into the database), death or discharge from hospital (defined as the time when the patient’s discharge from hospital was entered into the database), or transfer to another specialty. It also records emergency visits, patient demographics, and location during admission. The Laboratory Database records all laboratory tests and their results.

Study Cohort

The Patient Registry Database was used to identify all individuals who were admitted to the general medicine services between January 1 and December 31, 2015. This time frame was selected to ensure that data were complete and current. General medicine services were analyzed because they are collectively the largest inpatient specialty in the hospital.

Study Outcome

The primary outcome was discharge from hospital as determined from the Patient Registry Database. Patients who died or were transferred to another service were not counted as outcomes.

Covariables

The primary exposure variable was the consecutive number of days (including weekends) that a particular hospitalist rounded on patients on a particular general medicine service. This was measured using call schedules. Other covariates included tomorrow’s expected number of discharges (TEND) daily discharge probability and its components. The TEND model4 used patient factors (age, Laboratory Abnormality Physiological Score [LAPS]5 calculated at admission) and hospitalization factors (hospital campus and service, admission urgency, day of the week, ICU status) to predict the daily discharge probability. In a validation population, these daily discharge probabilities (when summed over a particular day) strongly predicted the daily number of discharges (adjusted R2 of 89.2% [P < .001], median relative difference between observed and expected number of discharges of only 1.4% [Interquartile range,IQR: −5.5% to 7.1%]). The expected annual death risk was determined using the HOMR-now! model.6 This model used routinely collected data available at patient admission regarding the patient (sex, life-table-estimated 1-year death risk, Charlson score, current living location, previous cancer clinic status, and number of emergency department visits in the previous year) and the hospitalization (urgency, service, and LAPS score). The model explained more than half of the total variability in death likelihood of death (Nagelkirke’s R2 value of 0.53),7 was highly discriminative (C-statistic 0.92), and accurately predicted death risk (calibration slope 0.98).

Analysis

Logistic generalized estimating equation (GEE) methods were used to model the adjusted daily discharge probability.⁸ Data in the analytical dataset were expressed in a patient-day format (each dataset row represented one day for a particular patient). This permitted the inclusion of time-dependent covariates and allowed the GEE model to cluster hospitalization days within patients.

Model construction started with the TEND daily discharge probability and the HOMR-now! expected annual death risk (both expressed as log-odds). Then, hospitalist continuity was entered as a time-dependent covariate (ie, its value changed every day). Linear, square root, and natural logarithm forms of physician continuity were examined to determine the best fit (determined using the QIC statistic⁹). Finally, individual components of the TEND model were also offered to the model with those which significantly improving fit kept in the model. The GEE model used an independent correlation structure since this minimized the QIC statistic in the base model. All covariates in the final daily discharge probability model were used in the hospital death model. Analyses were conducted using SAS 9.4 (Cary, NC).

There were 6,405 general medicine admissions involving 5208 patients and 38,967 patient-days between January 1 and December 31, 2015 (Appendix A). Patients were elderly and were evenly divided in terms of gender, with 85% of them being admitted from the community. Comorbidities were common (median coded Charlson score was 2), with 6.0% of patients known to our cancer clinic. The median length of stay was 4 days (IQR, 2–7), with 378 admissions (5.9%) ending in death and 121 admissions (1.9%) ending in a transfer to another service.

There were 41 different staff people having at least 1 day on service. The median total service by physicians was 9 weeks (IQR 1.8–10.9 weeks). Changes in hospitalist coverage were common; hospitalizations had a median of 1 (IQR 1–2) physician switches and a median of 1 (IQR 1–2) different physicians. However, patients spent a median of 100% (IQR 66.7%–100%] of their total hospitalization with their primary hospitalist. The median duration of individual physician “stints” on service was 5 days (IQR 2–7, range 1–42).

The TEND model accurately estimated daily discharge probability for the entire cohort with 5833 and 5718.6 observed and expected discharges, respectively, during 38,967 patient-days (O/E 1.02, 95% CI 0.99–1.05). Discharge probability increased as hospitalist continuity increased, but this was statistically significant only when hospitalist continuity exceeded 4 days. Other covariables also significantly influenced discharge probability (Appendix B).

After adjusting for important covariables (Appendix C), hospitalist continuity was significantly associated with daily discharge probability (Figure). Discharge probability increased linearly with increasing consecutive days that hospitalists treated patients. For each additional consecutive day with the same hospitalist, the adjusted daily odds increased by 2% (Adj-odds ratio [OR] 1.02, 95% CI 1.01–1.02, Appendix C). When the consecutive number of days that hospitalists remained on service increased from 1 to 28 days, the adjusted discharge probability for the average patient increased from 18.1% to 25.7%, respectively. Discharge was significantly influenced by other factors (Appendix C). Continuity did not influence the risk of death in hospital (Appendix D).

In a general medicine service at a large teaching hospital, this study found that greater hospitalist continuity was associated with a significantly increased adjusted daily discharge probability, increasing (in the average patient) from 18.1% to 25.7% when the consecutive number of hospitalist days on service increased from 1 to 28 days, respectively.

The study demonstrated some interesting findings. First, it shows that shifting patient care between physicians can significantly influence patient outcomes. This could be a function of incomplete transfer of knowledge between physicians, a phenomenon that should be expected given the extensive amount of information–both explicit and implicit–that physicians collect about particular patients during their hospitalization. Second, continuity of care could increase a physician’s and a patient’s confidence in clinical decision-making. Perhaps physicians are subconsciously more trusting of their instincts (and the decisions based on those instincts) when they have been on service for a while. It is also possible that patients more readily trust recommendations of a physician they have had throughout their stay. Finally, people wishing to decrease patient length of stay might consider minimizing the extent that hospitalists sign over patient care to colleagues.

Several issues should be noted when interpreting the results of the study. First, the study examined only patient discharge and death. These are by no means the only or the most important outcomes that might be influenced by hospitalist continuity. Second, this study was limited to a single service at a single center. Third, the analysis did not account for house-staff continuity. Since hospitalist and house-staff at the study hospital invariably switched at different times, it is unlikely that hospitalist continuity was a surrogate for house-staff continuity.

Disclosures

This study was supported by the Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada. The author has nothing to disclose.

In addition to treating patients, physicians frequently have other time commitments that could include administrative, teaching, research, and family duties. Inpatient medicine is particularly unforgiving to these nonclinical duties since patients have to be assessed on a daily basis. Because of this characteristic, it is not uncommon for inpatient care responsibility to be switched between physicians to create time for nonclinical duties and personal health.

In contrast to the ambulatory setting, the influence of physician continuity of care on inpatient outcomes has not been studied frequently. Studies of inpatient continuity have primarily focused on patient discharge (likely because of its objective nature) over the weekends (likely because weekend cross-coverage is common) and have reported conflicting results.^1-3 However, discontinuity of care is not isolated to the weekend since hospitalist-switches can occur at any time. In addition, expressing hospitalist continuity of care as a dichotomous variable (Was there weekend cross-coverage?) could incompletely express continuity since discharge likelihood might change with the consecutive number of days that a hospitalist is on service. This study measured the influence of hospitalist continuity throughout the patient’s hospitalization (rather than just the weekend) on daily patient discharge.

Study Setting and Databases Used for Analysis

The study was conducted at The Ottawa Hospital, Ontario, Canada, a 1000-bed teaching hospital with 2 campuses and the primary referral center in our region. The division of general internal medicine has 6 patient services (or “teams”) at two campuses led by a staff hospitalist (exclusively general internists), a senior medical resident (2nd year of training), and various numbers of interns and medical students. Staff hospitalists do not treat more than one patient service even on the weekends.

Patients are admitted to each service on a daily basis and almost exclusively from the emergency room. Assignment of patients is essentially random since all services have the same clinical expertise. At a particular campus, the number of patients assigned daily to each service is usually equivalent between teams. Patients almost never switch between teams but may be transferred to another specialty. The study was approved by our local research ethics board.

The Patient Registry Database records for each patient the date and time of admissions (defined as the moment that a patient’s admission request is entered into the database), death or discharge from hospital (defined as the time when the patient’s discharge from hospital was entered into the database), or transfer to another specialty. It also records emergency visits, patient demographics, and location during admission. The Laboratory Database records all laboratory tests and their results.

Study Cohort

The Patient Registry Database was used to identify all individuals who were admitted to the general medicine services between January 1 and December 31, 2015. This time frame was selected to ensure that data were complete and current. General medicine services were analyzed because they are collectively the largest inpatient specialty in the hospital.

Study Outcome

The primary outcome was discharge from hospital as determined from the Patient Registry Database. Patients who died or were transferred to another service were not counted as outcomes.

Covariables

The primary exposure variable was the consecutive number of days (including weekends) that a particular hospitalist rounded on patients on a particular general medicine service. This was measured using call schedules. Other covariates included tomorrow’s expected number of discharges (TEND) daily discharge probability and its components. The TEND model4 used patient factors (age, Laboratory Abnormality Physiological Score [LAPS]5 calculated at admission) and hospitalization factors (hospital campus and service, admission urgency, day of the week, ICU status) to predict the daily discharge probability. In a validation population, these daily discharge probabilities (when summed over a particular day) strongly predicted the daily number of discharges (adjusted R2 of 89.2% [P < .001], median relative difference between observed and expected number of discharges of only 1.4% [Interquartile range,IQR: −5.5% to 7.1%]). The expected annual death risk was determined using the HOMR-now! model.6 This model used routinely collected data available at patient admission regarding the patient (sex, life-table-estimated 1-year death risk, Charlson score, current living location, previous cancer clinic status, and number of emergency department visits in the previous year) and the hospitalization (urgency, service, and LAPS score). The model explained more than half of the total variability in death likelihood of death (Nagelkirke’s R2 value of 0.53),7 was highly discriminative (C-statistic 0.92), and accurately predicted death risk (calibration slope 0.98).

Analysis

Logistic generalized estimating equation (GEE) methods were used to model the adjusted daily discharge probability.⁸ Data in the analytical dataset were expressed in a patient-day format (each dataset row represented one day for a particular patient). This permitted the inclusion of time-dependent covariates and allowed the GEE model to cluster hospitalization days within patients.

Model construction started with the TEND daily discharge probability and the HOMR-now! expected annual death risk (both expressed as log-odds). Then, hospitalist continuity was entered as a time-dependent covariate (ie, its value changed every day). Linear, square root, and natural logarithm forms of physician continuity were examined to determine the best fit (determined using the QIC statistic⁹). Finally, individual components of the TEND model were also offered to the model with those which significantly improving fit kept in the model. The GEE model used an independent correlation structure since this minimized the QIC statistic in the base model. All covariates in the final daily discharge probability model were used in the hospital death model. Analyses were conducted using SAS 9.4 (Cary, NC).

There were 6,405 general medicine admissions involving 5208 patients and 38,967 patient-days between January 1 and December 31, 2015 (Appendix A). Patients were elderly and were evenly divided in terms of gender, with 85% of them being admitted from the community. Comorbidities were common (median coded Charlson score was 2), with 6.0% of patients known to our cancer clinic. The median length of stay was 4 days (IQR, 2–7), with 378 admissions (5.9%) ending in death and 121 admissions (1.9%) ending in a transfer to another service.

There were 41 different staff people having at least 1 day on service. The median total service by physicians was 9 weeks (IQR 1.8–10.9 weeks). Changes in hospitalist coverage were common; hospitalizations had a median of 1 (IQR 1–2) physician switches and a median of 1 (IQR 1–2) different physicians. However, patients spent a median of 100% (IQR 66.7%–100%] of their total hospitalization with their primary hospitalist. The median duration of individual physician “stints” on service was 5 days (IQR 2–7, range 1–42).

The TEND model accurately estimated daily discharge probability for the entire cohort with 5833 and 5718.6 observed and expected discharges, respectively, during 38,967 patient-days (O/E 1.02, 95% CI 0.99–1.05). Discharge probability increased as hospitalist continuity increased, but this was statistically significant only when hospitalist continuity exceeded 4 days. Other covariables also significantly influenced discharge probability (Appendix B).

After adjusting for important covariables (Appendix C), hospitalist continuity was significantly associated with daily discharge probability (Figure). Discharge probability increased linearly with increasing consecutive days that hospitalists treated patients. For each additional consecutive day with the same hospitalist, the adjusted daily odds increased by 2% (Adj-odds ratio [OR] 1.02, 95% CI 1.01–1.02, Appendix C). When the consecutive number of days that hospitalists remained on service increased from 1 to 28 days, the adjusted discharge probability for the average patient increased from 18.1% to 25.7%, respectively. Discharge was significantly influenced by other factors (Appendix C). Continuity did not influence the risk of death in hospital (Appendix D).

In a general medicine service at a large teaching hospital, this study found that greater hospitalist continuity was associated with a significantly increased adjusted daily discharge probability, increasing (in the average patient) from 18.1% to 25.7% when the consecutive number of hospitalist days on service increased from 1 to 28 days, respectively.

The study demonstrated some interesting findings. First, it shows that shifting patient care between physicians can significantly influence patient outcomes. This could be a function of incomplete transfer of knowledge between physicians, a phenomenon that should be expected given the extensive amount of information–both explicit and implicit–that physicians collect about particular patients during their hospitalization. Second, continuity of care could increase a physician’s and a patient’s confidence in clinical decision-making. Perhaps physicians are subconsciously more trusting of their instincts (and the decisions based on those instincts) when they have been on service for a while. It is also possible that patients more readily trust recommendations of a physician they have had throughout their stay. Finally, people wishing to decrease patient length of stay might consider minimizing the extent that hospitalists sign over patient care to colleagues.

Several issues should be noted when interpreting the results of the study. First, the study examined only patient discharge and death. These are by no means the only or the most important outcomes that might be influenced by hospitalist continuity. Second, this study was limited to a single service at a single center. Third, the analysis did not account for house-staff continuity. Since hospitalist and house-staff at the study hospital invariably switched at different times, it is unlikely that hospitalist continuity was a surrogate for house-staff continuity.

Disclosures

This study was supported by the Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada. The author has nothing to disclose.

In addition to treating patients, physicians frequently have other time commitments that could include administrative, teaching, research, and family duties. Inpatient medicine is particularly unforgiving to these nonclinical duties since patients have to be assessed on a daily basis. Because of this characteristic, it is not uncommon for inpatient care responsibility to be switched between physicians to create time for nonclinical duties and personal health.

In contrast to the ambulatory setting, the influence of physician continuity of care on inpatient outcomes has not been studied frequently. Studies of inpatient continuity have primarily focused on patient discharge (likely because of its objective nature) over the weekends (likely because weekend cross-coverage is common) and have reported conflicting results.^1-3 However, discontinuity of care is not isolated to the weekend since hospitalist-switches can occur at any time. In addition, expressing hospitalist continuity of care as a dichotomous variable (Was there weekend cross-coverage?) could incompletely express continuity since discharge likelihood might change with the consecutive number of days that a hospitalist is on service. This study measured the influence of hospitalist continuity throughout the patient’s hospitalization (rather than just the weekend) on daily patient discharge.

Study Setting and Databases Used for Analysis

The study was conducted at The Ottawa Hospital, Ontario, Canada, a 1000-bed teaching hospital with 2 campuses and the primary referral center in our region. The division of general internal medicine has 6 patient services (or “teams”) at two campuses led by a staff hospitalist (exclusively general internists), a senior medical resident (2nd year of training), and various numbers of interns and medical students. Staff hospitalists do not treat more than one patient service even on the weekends.

Patients are admitted to each service on a daily basis and almost exclusively from the emergency room. Assignment of patients is essentially random since all services have the same clinical expertise. At a particular campus, the number of patients assigned daily to each service is usually equivalent between teams. Patients almost never switch between teams but may be transferred to another specialty. The study was approved by our local research ethics board.

The Patient Registry Database records for each patient the date and time of admissions (defined as the moment that a patient’s admission request is entered into the database), death or discharge from hospital (defined as the time when the patient’s discharge from hospital was entered into the database), or transfer to another specialty. It also records emergency visits, patient demographics, and location during admission. The Laboratory Database records all laboratory tests and their results.

Study Cohort

The Patient Registry Database was used to identify all individuals who were admitted to the general medicine services between January 1 and December 31, 2015. This time frame was selected to ensure that data were complete and current. General medicine services were analyzed because they are collectively the largest inpatient specialty in the hospital.

Study Outcome

The primary outcome was discharge from hospital as determined from the Patient Registry Database. Patients who died or were transferred to another service were not counted as outcomes.

Covariables

The primary exposure variable was the consecutive number of days (including weekends) that a particular hospitalist rounded on patients on a particular general medicine service. This was measured using call schedules. Other covariates included tomorrow’s expected number of discharges (TEND) daily discharge probability and its components. The TEND model4 used patient factors (age, Laboratory Abnormality Physiological Score [LAPS]5 calculated at admission) and hospitalization factors (hospital campus and service, admission urgency, day of the week, ICU status) to predict the daily discharge probability. In a validation population, these daily discharge probabilities (when summed over a particular day) strongly predicted the daily number of discharges (adjusted R2 of 89.2% [P < .001], median relative difference between observed and expected number of discharges of only 1.4% [Interquartile range,IQR: −5.5% to 7.1%]). The expected annual death risk was determined using the HOMR-now! model.6 This model used routinely collected data available at patient admission regarding the patient (sex, life-table-estimated 1-year death risk, Charlson score, current living location, previous cancer clinic status, and number of emergency department visits in the previous year) and the hospitalization (urgency, service, and LAPS score). The model explained more than half of the total variability in death likelihood of death (Nagelkirke’s R2 value of 0.53),7 was highly discriminative (C-statistic 0.92), and accurately predicted death risk (calibration slope 0.98).

Analysis

Logistic generalized estimating equation (GEE) methods were used to model the adjusted daily discharge probability.⁸ Data in the analytical dataset were expressed in a patient-day format (each dataset row represented one day for a particular patient). This permitted the inclusion of time-dependent covariates and allowed the GEE model to cluster hospitalization days within patients.

Model construction started with the TEND daily discharge probability and the HOMR-now! expected annual death risk (both expressed as log-odds). Then, hospitalist continuity was entered as a time-dependent covariate (ie, its value changed every day). Linear, square root, and natural logarithm forms of physician continuity were examined to determine the best fit (determined using the QIC statistic⁹). Finally, individual components of the TEND model were also offered to the model with those which significantly improving fit kept in the model. The GEE model used an independent correlation structure since this minimized the QIC statistic in the base model. All covariates in the final daily discharge probability model were used in the hospital death model. Analyses were conducted using SAS 9.4 (Cary, NC).

There were 6,405 general medicine admissions involving 5208 patients and 38,967 patient-days between January 1 and December 31, 2015 (Appendix A). Patients were elderly and were evenly divided in terms of gender, with 85% of them being admitted from the community. Comorbidities were common (median coded Charlson score was 2), with 6.0% of patients known to our cancer clinic. The median length of stay was 4 days (IQR, 2–7), with 378 admissions (5.9%) ending in death and 121 admissions (1.9%) ending in a transfer to another service.

There were 41 different staff people having at least 1 day on service. The median total service by physicians was 9 weeks (IQR 1.8–10.9 weeks). Changes in hospitalist coverage were common; hospitalizations had a median of 1 (IQR 1–2) physician switches and a median of 1 (IQR 1–2) different physicians. However, patients spent a median of 100% (IQR 66.7%–100%] of their total hospitalization with their primary hospitalist. The median duration of individual physician “stints” on service was 5 days (IQR 2–7, range 1–42).

The TEND model accurately estimated daily discharge probability for the entire cohort with 5833 and 5718.6 observed and expected discharges, respectively, during 38,967 patient-days (O/E 1.02, 95% CI 0.99–1.05). Discharge probability increased as hospitalist continuity increased, but this was statistically significant only when hospitalist continuity exceeded 4 days. Other covariables also significantly influenced discharge probability (Appendix B).

After adjusting for important covariables (Appendix C), hospitalist continuity was significantly associated with daily discharge probability (Figure). Discharge probability increased linearly with increasing consecutive days that hospitalists treated patients. For each additional consecutive day with the same hospitalist, the adjusted daily odds increased by 2% (Adj-odds ratio [OR] 1.02, 95% CI 1.01–1.02, Appendix C). When the consecutive number of days that hospitalists remained on service increased from 1 to 28 days, the adjusted discharge probability for the average patient increased from 18.1% to 25.7%, respectively. Discharge was significantly influenced by other factors (Appendix C). Continuity did not influence the risk of death in hospital (Appendix D).

In a general medicine service at a large teaching hospital, this study found that greater hospitalist continuity was associated with a significantly increased adjusted daily discharge probability, increasing (in the average patient) from 18.1% to 25.7% when the consecutive number of hospitalist days on service increased from 1 to 28 days, respectively.

The study demonstrated some interesting findings. First, it shows that shifting patient care between physicians can significantly influence patient outcomes. This could be a function of incomplete transfer of knowledge between physicians, a phenomenon that should be expected given the extensive amount of information–both explicit and implicit–that physicians collect about particular patients during their hospitalization. Second, continuity of care could increase a physician’s and a patient’s confidence in clinical decision-making. Perhaps physicians are subconsciously more trusting of their instincts (and the decisions based on those instincts) when they have been on service for a while. It is also possible that patients more readily trust recommendations of a physician they have had throughout their stay. Finally, people wishing to decrease patient length of stay might consider minimizing the extent that hospitalists sign over patient care to colleagues.

Several issues should be noted when interpreting the results of the study. First, the study examined only patient discharge and death. These are by no means the only or the most important outcomes that might be influenced by hospitalist continuity. Second, this study was limited to a single service at a single center. Third, the analysis did not account for house-staff continuity. Since hospitalist and house-staff at the study hospital invariably switched at different times, it is unlikely that hospitalist continuity was a surrogate for house-staff continuity.

Disclosures

This study was supported by the Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada. The author has nothing to disclose.

SAN DIEGO – Between 2009 and 2015, the number of emergency department visits related to mental health increased 56% among children and 41% among adults, results from an analysis of national hospital data showed.

[email protected]

SOURCE: Santillanes G et al. Ann Emerg Med. 2018 Oct. doi. 10.1016/j.annemergmed.2018.08.050.

SAN DIEGO – Between 2009 and 2015, the number of emergency department visits related to mental health increased 56% among children and 41% among adults, results from an analysis of national hospital data showed.

[email protected]

SOURCE: Santillanes G et al. Ann Emerg Med. 2018 Oct. doi. 10.1016/j.annemergmed.2018.08.050.

SAN DIEGO – Between 2009 and 2015, the number of emergency department visits related to mental health increased 56% among children and 41% among adults, results from an analysis of national hospital data showed.

[email protected]

SOURCE: Santillanes G et al. Ann Emerg Med. 2018 Oct. doi. 10.1016/j.annemergmed.2018.08.050.

Click here to access the October 2018 Digital Edition.

Table of Contents

• Comparison of Cardiovascular Outcomes Between Statin Monotherapy and Fish Oil and Statin Combination Therapy in a Veteran Population
• Improving Team-Based Care Coordination Delivery and Documentation in the Health Record
• An Overview of Pharmacotherapy Options for Alcohol Use Disorder
• An Imposter Twice Over: A Case of IgG4-Related Disease
• Happy Federal New Year
• FDA Update

Click here to access the October 2018 Digital Edition.

Table of Contents

• Comparison of Cardiovascular Outcomes Between Statin Monotherapy and Fish Oil and Statin Combination Therapy in a Veteran Population
• Improving Team-Based Care Coordination Delivery and Documentation in the Health Record
• An Overview of Pharmacotherapy Options for Alcohol Use Disorder
• An Imposter Twice Over: A Case of IgG4-Related Disease
• Happy Federal New Year
• FDA Update

Click here to access the October 2018 Digital Edition.

Table of Contents

• Comparison of Cardiovascular Outcomes Between Statin Monotherapy and Fish Oil and Statin Combination Therapy in a Veteran Population
• Improving Team-Based Care Coordination Delivery and Documentation in the Health Record
• An Overview of Pharmacotherapy Options for Alcohol Use Disorder
• An Imposter Twice Over: A Case of IgG4-Related Disease
• Happy Federal New Year
• FDA Update