User login
In the current health care environment, hospitals are constantly challenged to improve quality metrics and deliver better health care outcomes. One means to achieving quality improvement is through the use of order sets, groups of related orders that a health care provider (HCP) can place with either a few keystrokes or mouse clicks.1
Historically, design of order sets has largely focused on clicking checkboxes containing evidence-based practices. According to Bates and colleagues and the Institute for Safe Medication Practices, incorporating evidence-based medicine (EBM) into order sets is not by itself sufficient.2,3Execution of proper design coupled with simplicity and provider efficiency is paramount to HCP buy-in, increased likelihood of order set adherence, and to potentially better outcomes.
In this article, we outline advancements in order set design. These improvements increase provider efficiency and ease of use; incorporate human factors engineering (HFE); apply failure mode and effects analysis; and include EBM.
Methods
An inpatient nicotine replacement therapy (NRT) order was developed as part of a multifaceted solution to improve tobacco cessation care at the James A. Haley Veterans’ Hospital (JAHVH) in Tampa, Florida, a complexity level 1a facility. This NRT order set used the 4-step order set design framework the authors’ developed (for additional information about the NRT order set, contact the authors). We distinguish order set design technique between 2 different inpatient NRT order sets. The first order set in the comparison (Figure 1) is an inpatient NRT order set of unknown origin—it is common for US Department of Veterans Affairs (VA) medical facilities to share order sets and other resources. The second order set (Figure 2) is an inpatient NRT order set we designed using our 4-step process for comparison in this article. No institutional review board approval was required as this work met criteria for operational improvement activities exempt from ethics review.
Justin Iannello, DO, MBA, was the team leader and developer of the 4-step order set design technique. The intervention team consisted of 4 internal medicine physicians with expertise in quality improvement and patient safety: 1 certified professional in patient safety and certified as a Lean Six Sigma Black Belt; 2 physicians certified as Lean Six Sigma Black Belts; and 1 physician certified as a Lean Six Sigma Green Belt. Two inpatient clinical pharmacists and 1 quality management specialist also were involved in its development.
Development of a new NRT order set was felt to be an integral part of the tobacco cessation care delivery process. An NRT order set perceived by users as value-added required a solution that merged EBM with standardization and applied quality improvement principles. The result was an approach to order set design that focused on 4 key questions: Is the order set efficient and easy to use/navigate? Is human factors engineering incorporated? Is failure mode and effects analysis applied? Are evidence-based practices included?
Ease of Use and Navigation
Implementing an order set that is efficient and easy to use or navigate seems straightforward but can be difficult to execute. Figure 1 shows many detailed options consisting of different combinations of nicotine patches, lozenges, and gum. Also included are oral tobacco cessation options (bupropion and varenicline). Although more options may seem better, confusion about appropriate medication selection can occur.
According to Heath and Heath, too many options can result in lack of action.4 For example, Heath and Heath discuss a food store that offered 6 free samples of different jams on one day and 24 jams the following day. The customers who sampled 6 different types of jam were 10 times more likely to buy jam. The authors concluded that the more options available, the more difficulty a potential buyer has in deciding on a course of action.4
In clinical situations where a HCP is using an order set, the number of options can mean the difference between use vs avoidance if the choices are overwhelming. HCPs process layers of detail every day when creating differential diagnoses and treatment plans. While that level of detail is necessary clinically, that same level of detail included in orders sets can create challenges for HCPs.
Figure 2 advances the order set in Figure 1 by providing a simpler and cleaner design, so HCPs can more easily review and process the information. This order set design minimizes the number of options available to help users make the right decision, focusing on value for the appropriate setting and audience. In other words, order sets should not be a “one size fits all” approach.
Order sets should be tailored to the appropriate clinical setting (eg, inpatient acute care, outpatient clinic setting, etc) and HCP (eg, hospitalist, tobacco cessation specialist, etc). We are comparing NRT order sets designed for HCPs who do not routinely prescribe oral tobacco cessation products in the inpatient setting. When possible, autogenerated bundle orders should also be used according to evidence-based recommendations (such as nicotine patch tapers) for ease of use and further simplification of order sets.
Finally, usability testing known as “evaluating a product or service by testing it with representative users” helps further refine an order set.5Usability testing should be applied during all phases of order set development with end user(s) as it helps identify problems with order set design prior to implementation. By applying usability testing, the order set becomes more meaningful and valued by the user.
Human Factors Engineering
HFE is “the study of all the factors that make it easier to do the work in the right way.”6 HFE seeks to identify, align, and apply processes for people and the world within which they live and work to promote safe and efficient practices, especially in relation to the technology and physical design features in their work environment.6
The average American adult makes about 35,000 decisions per day.7 Thus, there is potential for error at any moment. Design that does not take HFE into account can be dangerous. For example, when tube feed and IV line connectors look similar and are compatible, patients may inadvertently receive food administered directly into their bloodstream.8
HFE can and should be applied to order sets. Everything from the look, feel, and verbiage of an order set affects potential outcomes. For example, consider the impact even seemingly minor modifications can have on outcomes simply by guiding users in a different way: Figure 1 provides NRT options based on cigarette use per day, whereas Figure 2 conveys pack use per day in relation to the equivalent number of cigarettes used daily. These differences may seem small; however, it helps guide users to the right choice when considering that health care providers have been historically trained on social history gathering that emphasizes packs per day and pack-years.
Failure Mode and Effects Analysis
Failure mode and effects analysis (FMEA) is “a structured way to identify and address potential problems, or failures and their resulting effects on the system or process before an adverse event occurs.”9 The benefit of an order set must be weighed against the risk during development. FMEA should be applied during order set design to assess and limit risk just as with any other clinical care process.
FMEA examines both level of risk and frequency of risk occurrence associated with a new proposed process. For example, let’s evaluate an order set designed for pain control after surgery that consists of multiple high-risk opioids along with antihistamine medications for as-needed itch relief (a non-life-threatening adverse event (AE) of opioids well known by the medical community). An interdisciplinary FMEA team consisting of subject matter experts may examine how the process should flow in step-by-step detail and then discuss the benefit of a process and risk for potential error. A FMEA team would then analyze what could go wrong with each part of the process and assign a level of risk and risk frequency for various steps in the process, and then decide that certain steps should be modified or eliminated. Perhaps after FMEA, a facility might conclude that the risk of serious complications is high when you combine opioid use with antihistamine medications. The facility could decide to remove antihistamine medications from an order set if it is determined that risks outweigh benefits. While a root cause analysis might identify the cause of an AE after order set use, these situations can be prevented with FMEA.
When applying FMEA to Figure 1, while bupropion is known as an evidence-based oral tobacco cessation option, there is the possibility that bupropion could be inadvertently prescribed from the order set in a hospitalized patient with alcohol withdrawal and withdrawal seizure history. These potentially dangerous situations can be avoided with FMEA. Thus, although bupropion may be evidence-based for NRT, decisions regarding order set design using EBM alone are insufficient.
The practitioner must consider possible unintended consequences within order sets and target treatment options to the appropriate setting and audience. Although Figure 1 may appear to be more inclusive, the interdisciplinary committee designing the inpatient NRT order set felt there was heightened risk with introducing bupropion in Figure 1 and decided the risk would be lowered by removing bupropion from the redesigned NRT order set (Figure 2). In addition to the goal of balancing availability of NRT options with acceptable risk, Figure 2 also focused on building an NRT order set most applicable to the inpatient setting.
Including Evidence-Based Practices
EBM has become a routine part of clinical decision making. Therefore, including EBM in order set design is vital. EBM for NRT has demonstrated that combination therapy is more effective than is monotherapy to help tobacco users quit. Incremental doses of NRT are recommended for patients who use tobacco more frequently.10
As shown in Figures 1 and 2, both order set designs incorporate EBM for NRT. Although the importance of implementing EBM is evident, critical factors, such as HFE and FMEA make a difference with well-designed order sets.
Results
The 4-step order set design technique was used during development of an inpatient NRT order set at the JAHVH. Results for the inpatient Joint Commission Tobacco Treatment Measures were obtained from the Veterans Health Administration quality metric reporting system known as Strategic Analytics for Improvement and Learning (SAIL). SAIL performance measure outcomes, which include the inpatient Joint Commission Tobacco Treatment Measures, are derived from chart reviews conducted by the External Peer Review Program. Outcomes demonstrated that TOB-2 and TOB-3 (2 inpatient Joint Commission Tobacco Treatment Measures) known as tob20 and tob40, respectively, within SAIL improved by more than 300% after development of an NRT order set using the 4-step order set design framework along with implementation of a multifaceted tobacco cessation care delivery system at JAHVH.
Discussion
While the overall tobacco cessation care delivery system contributed to improved outcomes with the inpatient Joint Commission Tobacco Treatment Measures at JAHVH, the NRT order set was a cornerstone of the design. Although using our order set design technique does not necessarily guarantee successful outcomes, we believe using the 4-step order set design process increases the value of order sets and has potential to improve quality outcomes.
Limitations
Although improved outcomes following implementation of our NRT order set suggest correlation, causation cannot be proven. Also while the NRT order set is believed to have helped tremendously with outcomes, the entire tobacco cessation care delivery system at JAHVH contributed to the results. In addition, the inpatient Joint Commission Tobacco Treatment Measures help improve processes for tobacco cessation care. However, we are uncertain whether the results of our improvement efforts helped patients stop tobacco use. Further studies are needed to determine impact on population health. Finally, our results were based on improvement work done at a single center. Further studies are necessary to see whether results are reproducible.
Conclusion
There was significant improvement with the inpatient Joint Commission Tobacco Treatment Measures outcomes following development of a tobacco cessation care delivery system that included design of an inpatient NRT order set using a 4-step process we developed. This 4-step structure includes emphasis on efficiency and ease of use; human factors engineering; failure mode and effects analysis; and incorporation of evidence-based medicine (Box.) Postimplementation results showed improvement of the inpatient Joint Commission Tobacco Treatment Measures by greater than 3-fold at a single hospital.
The next steps for this initiative include testing the 4-step order set design process in multiple clinical settings to determine the effectiveness of this approach in other areas of clinical care.
1. Order set. http://clinfowiki.org/wiki/index.php/Order_set. Updated October 15, 2015. Accessed August 30, 2019.
2. Bates DW, Kuperman GJ, Wang S, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc. 2003;10(6):523-530.
3. Institute for Safe Medication Practices. Guidelines for standard order sets. https://www.ismp.org/tools/guidelines/standardordersets.pdf. Published January 12, 2010. Accessed August 30, 2019.
4. Heath C, Heath D. Switch: How to Change Things When Change Is Hard. New York, NY: Crown Business; 2010:50-51.
5. US Department of Health and Human Services. Usability testing. https://www.usability.gov/how-to-and-tools/methods/usability-testing.html. Accessed August 30, 2019.
6. World Health Organization. What is human factors and why is it important to patient safety? www.who.int/patientsafety/education/curriculum/who_mc_topic-2.pdf. Accessed August 30, 2019.
7. Sollisch J. The cure for decision fatigue. Wall Street Journal. June 10, 2016. https://www.wsj.com/articles/the-cure-for-decision-fatigue-1465596928. Accessed August 30, 2019.
8. ECRI Institute. Implementing the ENFit initiative for preventing enteral tubing misconnections. https://www.ecri.org/components/HDJournal/Pages/ENFit-for-Preventing-Enteral-Tubing-Misconnections.aspx. Published March 29, 2017. Accessed August 30, 2019.
9. Guidance for performing failure mode and effects analysis with performance improvement projects. https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/QAPI/downloads/GuidanceForFMEA.pdf. Accessed August 30, 2019.
10. Diefanbach LJ, Smith PO, Nashelsky JM, Lindbloom E. What is the most effective nicotine replacement therapy? J Fam Pract. 2003;52(6):492-497.
In the current health care environment, hospitals are constantly challenged to improve quality metrics and deliver better health care outcomes. One means to achieving quality improvement is through the use of order sets, groups of related orders that a health care provider (HCP) can place with either a few keystrokes or mouse clicks.1
Historically, design of order sets has largely focused on clicking checkboxes containing evidence-based practices. According to Bates and colleagues and the Institute for Safe Medication Practices, incorporating evidence-based medicine (EBM) into order sets is not by itself sufficient.2,3Execution of proper design coupled with simplicity and provider efficiency is paramount to HCP buy-in, increased likelihood of order set adherence, and to potentially better outcomes.
In this article, we outline advancements in order set design. These improvements increase provider efficiency and ease of use; incorporate human factors engineering (HFE); apply failure mode and effects analysis; and include EBM.
Methods
An inpatient nicotine replacement therapy (NRT) order was developed as part of a multifaceted solution to improve tobacco cessation care at the James A. Haley Veterans’ Hospital (JAHVH) in Tampa, Florida, a complexity level 1a facility. This NRT order set used the 4-step order set design framework the authors’ developed (for additional information about the NRT order set, contact the authors). We distinguish order set design technique between 2 different inpatient NRT order sets. The first order set in the comparison (Figure 1) is an inpatient NRT order set of unknown origin—it is common for US Department of Veterans Affairs (VA) medical facilities to share order sets and other resources. The second order set (Figure 2) is an inpatient NRT order set we designed using our 4-step process for comparison in this article. No institutional review board approval was required as this work met criteria for operational improvement activities exempt from ethics review.
Justin Iannello, DO, MBA, was the team leader and developer of the 4-step order set design technique. The intervention team consisted of 4 internal medicine physicians with expertise in quality improvement and patient safety: 1 certified professional in patient safety and certified as a Lean Six Sigma Black Belt; 2 physicians certified as Lean Six Sigma Black Belts; and 1 physician certified as a Lean Six Sigma Green Belt. Two inpatient clinical pharmacists and 1 quality management specialist also were involved in its development.
Development of a new NRT order set was felt to be an integral part of the tobacco cessation care delivery process. An NRT order set perceived by users as value-added required a solution that merged EBM with standardization and applied quality improvement principles. The result was an approach to order set design that focused on 4 key questions: Is the order set efficient and easy to use/navigate? Is human factors engineering incorporated? Is failure mode and effects analysis applied? Are evidence-based practices included?
Ease of Use and Navigation
Implementing an order set that is efficient and easy to use or navigate seems straightforward but can be difficult to execute. Figure 1 shows many detailed options consisting of different combinations of nicotine patches, lozenges, and gum. Also included are oral tobacco cessation options (bupropion and varenicline). Although more options may seem better, confusion about appropriate medication selection can occur.
According to Heath and Heath, too many options can result in lack of action.4 For example, Heath and Heath discuss a food store that offered 6 free samples of different jams on one day and 24 jams the following day. The customers who sampled 6 different types of jam were 10 times more likely to buy jam. The authors concluded that the more options available, the more difficulty a potential buyer has in deciding on a course of action.4
In clinical situations where a HCP is using an order set, the number of options can mean the difference between use vs avoidance if the choices are overwhelming. HCPs process layers of detail every day when creating differential diagnoses and treatment plans. While that level of detail is necessary clinically, that same level of detail included in orders sets can create challenges for HCPs.
Figure 2 advances the order set in Figure 1 by providing a simpler and cleaner design, so HCPs can more easily review and process the information. This order set design minimizes the number of options available to help users make the right decision, focusing on value for the appropriate setting and audience. In other words, order sets should not be a “one size fits all” approach.
Order sets should be tailored to the appropriate clinical setting (eg, inpatient acute care, outpatient clinic setting, etc) and HCP (eg, hospitalist, tobacco cessation specialist, etc). We are comparing NRT order sets designed for HCPs who do not routinely prescribe oral tobacco cessation products in the inpatient setting. When possible, autogenerated bundle orders should also be used according to evidence-based recommendations (such as nicotine patch tapers) for ease of use and further simplification of order sets.
Finally, usability testing known as “evaluating a product or service by testing it with representative users” helps further refine an order set.5Usability testing should be applied during all phases of order set development with end user(s) as it helps identify problems with order set design prior to implementation. By applying usability testing, the order set becomes more meaningful and valued by the user.
Human Factors Engineering
HFE is “the study of all the factors that make it easier to do the work in the right way.”6 HFE seeks to identify, align, and apply processes for people and the world within which they live and work to promote safe and efficient practices, especially in relation to the technology and physical design features in their work environment.6
The average American adult makes about 35,000 decisions per day.7 Thus, there is potential for error at any moment. Design that does not take HFE into account can be dangerous. For example, when tube feed and IV line connectors look similar and are compatible, patients may inadvertently receive food administered directly into their bloodstream.8
HFE can and should be applied to order sets. Everything from the look, feel, and verbiage of an order set affects potential outcomes. For example, consider the impact even seemingly minor modifications can have on outcomes simply by guiding users in a different way: Figure 1 provides NRT options based on cigarette use per day, whereas Figure 2 conveys pack use per day in relation to the equivalent number of cigarettes used daily. These differences may seem small; however, it helps guide users to the right choice when considering that health care providers have been historically trained on social history gathering that emphasizes packs per day and pack-years.
Failure Mode and Effects Analysis
Failure mode and effects analysis (FMEA) is “a structured way to identify and address potential problems, or failures and their resulting effects on the system or process before an adverse event occurs.”9 The benefit of an order set must be weighed against the risk during development. FMEA should be applied during order set design to assess and limit risk just as with any other clinical care process.
FMEA examines both level of risk and frequency of risk occurrence associated with a new proposed process. For example, let’s evaluate an order set designed for pain control after surgery that consists of multiple high-risk opioids along with antihistamine medications for as-needed itch relief (a non-life-threatening adverse event (AE) of opioids well known by the medical community). An interdisciplinary FMEA team consisting of subject matter experts may examine how the process should flow in step-by-step detail and then discuss the benefit of a process and risk for potential error. A FMEA team would then analyze what could go wrong with each part of the process and assign a level of risk and risk frequency for various steps in the process, and then decide that certain steps should be modified or eliminated. Perhaps after FMEA, a facility might conclude that the risk of serious complications is high when you combine opioid use with antihistamine medications. The facility could decide to remove antihistamine medications from an order set if it is determined that risks outweigh benefits. While a root cause analysis might identify the cause of an AE after order set use, these situations can be prevented with FMEA.
When applying FMEA to Figure 1, while bupropion is known as an evidence-based oral tobacco cessation option, there is the possibility that bupropion could be inadvertently prescribed from the order set in a hospitalized patient with alcohol withdrawal and withdrawal seizure history. These potentially dangerous situations can be avoided with FMEA. Thus, although bupropion may be evidence-based for NRT, decisions regarding order set design using EBM alone are insufficient.
The practitioner must consider possible unintended consequences within order sets and target treatment options to the appropriate setting and audience. Although Figure 1 may appear to be more inclusive, the interdisciplinary committee designing the inpatient NRT order set felt there was heightened risk with introducing bupropion in Figure 1 and decided the risk would be lowered by removing bupropion from the redesigned NRT order set (Figure 2). In addition to the goal of balancing availability of NRT options with acceptable risk, Figure 2 also focused on building an NRT order set most applicable to the inpatient setting.
Including Evidence-Based Practices
EBM has become a routine part of clinical decision making. Therefore, including EBM in order set design is vital. EBM for NRT has demonstrated that combination therapy is more effective than is monotherapy to help tobacco users quit. Incremental doses of NRT are recommended for patients who use tobacco more frequently.10
As shown in Figures 1 and 2, both order set designs incorporate EBM for NRT. Although the importance of implementing EBM is evident, critical factors, such as HFE and FMEA make a difference with well-designed order sets.
Results
The 4-step order set design technique was used during development of an inpatient NRT order set at the JAHVH. Results for the inpatient Joint Commission Tobacco Treatment Measures were obtained from the Veterans Health Administration quality metric reporting system known as Strategic Analytics for Improvement and Learning (SAIL). SAIL performance measure outcomes, which include the inpatient Joint Commission Tobacco Treatment Measures, are derived from chart reviews conducted by the External Peer Review Program. Outcomes demonstrated that TOB-2 and TOB-3 (2 inpatient Joint Commission Tobacco Treatment Measures) known as tob20 and tob40, respectively, within SAIL improved by more than 300% after development of an NRT order set using the 4-step order set design framework along with implementation of a multifaceted tobacco cessation care delivery system at JAHVH.
Discussion
While the overall tobacco cessation care delivery system contributed to improved outcomes with the inpatient Joint Commission Tobacco Treatment Measures at JAHVH, the NRT order set was a cornerstone of the design. Although using our order set design technique does not necessarily guarantee successful outcomes, we believe using the 4-step order set design process increases the value of order sets and has potential to improve quality outcomes.
Limitations
Although improved outcomes following implementation of our NRT order set suggest correlation, causation cannot be proven. Also while the NRT order set is believed to have helped tremendously with outcomes, the entire tobacco cessation care delivery system at JAHVH contributed to the results. In addition, the inpatient Joint Commission Tobacco Treatment Measures help improve processes for tobacco cessation care. However, we are uncertain whether the results of our improvement efforts helped patients stop tobacco use. Further studies are needed to determine impact on population health. Finally, our results were based on improvement work done at a single center. Further studies are necessary to see whether results are reproducible.
Conclusion
There was significant improvement with the inpatient Joint Commission Tobacco Treatment Measures outcomes following development of a tobacco cessation care delivery system that included design of an inpatient NRT order set using a 4-step process we developed. This 4-step structure includes emphasis on efficiency and ease of use; human factors engineering; failure mode and effects analysis; and incorporation of evidence-based medicine (Box.) Postimplementation results showed improvement of the inpatient Joint Commission Tobacco Treatment Measures by greater than 3-fold at a single hospital.
The next steps for this initiative include testing the 4-step order set design process in multiple clinical settings to determine the effectiveness of this approach in other areas of clinical care.
In the current health care environment, hospitals are constantly challenged to improve quality metrics and deliver better health care outcomes. One means to achieving quality improvement is through the use of order sets, groups of related orders that a health care provider (HCP) can place with either a few keystrokes or mouse clicks.1
Historically, design of order sets has largely focused on clicking checkboxes containing evidence-based practices. According to Bates and colleagues and the Institute for Safe Medication Practices, incorporating evidence-based medicine (EBM) into order sets is not by itself sufficient.2,3Execution of proper design coupled with simplicity and provider efficiency is paramount to HCP buy-in, increased likelihood of order set adherence, and to potentially better outcomes.
In this article, we outline advancements in order set design. These improvements increase provider efficiency and ease of use; incorporate human factors engineering (HFE); apply failure mode and effects analysis; and include EBM.
Methods
An inpatient nicotine replacement therapy (NRT) order was developed as part of a multifaceted solution to improve tobacco cessation care at the James A. Haley Veterans’ Hospital (JAHVH) in Tampa, Florida, a complexity level 1a facility. This NRT order set used the 4-step order set design framework the authors’ developed (for additional information about the NRT order set, contact the authors). We distinguish order set design technique between 2 different inpatient NRT order sets. The first order set in the comparison (Figure 1) is an inpatient NRT order set of unknown origin—it is common for US Department of Veterans Affairs (VA) medical facilities to share order sets and other resources. The second order set (Figure 2) is an inpatient NRT order set we designed using our 4-step process for comparison in this article. No institutional review board approval was required as this work met criteria for operational improvement activities exempt from ethics review.
Justin Iannello, DO, MBA, was the team leader and developer of the 4-step order set design technique. The intervention team consisted of 4 internal medicine physicians with expertise in quality improvement and patient safety: 1 certified professional in patient safety and certified as a Lean Six Sigma Black Belt; 2 physicians certified as Lean Six Sigma Black Belts; and 1 physician certified as a Lean Six Sigma Green Belt. Two inpatient clinical pharmacists and 1 quality management specialist also were involved in its development.
Development of a new NRT order set was felt to be an integral part of the tobacco cessation care delivery process. An NRT order set perceived by users as value-added required a solution that merged EBM with standardization and applied quality improvement principles. The result was an approach to order set design that focused on 4 key questions: Is the order set efficient and easy to use/navigate? Is human factors engineering incorporated? Is failure mode and effects analysis applied? Are evidence-based practices included?
Ease of Use and Navigation
Implementing an order set that is efficient and easy to use or navigate seems straightforward but can be difficult to execute. Figure 1 shows many detailed options consisting of different combinations of nicotine patches, lozenges, and gum. Also included are oral tobacco cessation options (bupropion and varenicline). Although more options may seem better, confusion about appropriate medication selection can occur.
According to Heath and Heath, too many options can result in lack of action.4 For example, Heath and Heath discuss a food store that offered 6 free samples of different jams on one day and 24 jams the following day. The customers who sampled 6 different types of jam were 10 times more likely to buy jam. The authors concluded that the more options available, the more difficulty a potential buyer has in deciding on a course of action.4
In clinical situations where a HCP is using an order set, the number of options can mean the difference between use vs avoidance if the choices are overwhelming. HCPs process layers of detail every day when creating differential diagnoses and treatment plans. While that level of detail is necessary clinically, that same level of detail included in orders sets can create challenges for HCPs.
Figure 2 advances the order set in Figure 1 by providing a simpler and cleaner design, so HCPs can more easily review and process the information. This order set design minimizes the number of options available to help users make the right decision, focusing on value for the appropriate setting and audience. In other words, order sets should not be a “one size fits all” approach.
Order sets should be tailored to the appropriate clinical setting (eg, inpatient acute care, outpatient clinic setting, etc) and HCP (eg, hospitalist, tobacco cessation specialist, etc). We are comparing NRT order sets designed for HCPs who do not routinely prescribe oral tobacco cessation products in the inpatient setting. When possible, autogenerated bundle orders should also be used according to evidence-based recommendations (such as nicotine patch tapers) for ease of use and further simplification of order sets.
Finally, usability testing known as “evaluating a product or service by testing it with representative users” helps further refine an order set.5Usability testing should be applied during all phases of order set development with end user(s) as it helps identify problems with order set design prior to implementation. By applying usability testing, the order set becomes more meaningful and valued by the user.
Human Factors Engineering
HFE is “the study of all the factors that make it easier to do the work in the right way.”6 HFE seeks to identify, align, and apply processes for people and the world within which they live and work to promote safe and efficient practices, especially in relation to the technology and physical design features in their work environment.6
The average American adult makes about 35,000 decisions per day.7 Thus, there is potential for error at any moment. Design that does not take HFE into account can be dangerous. For example, when tube feed and IV line connectors look similar and are compatible, patients may inadvertently receive food administered directly into their bloodstream.8
HFE can and should be applied to order sets. Everything from the look, feel, and verbiage of an order set affects potential outcomes. For example, consider the impact even seemingly minor modifications can have on outcomes simply by guiding users in a different way: Figure 1 provides NRT options based on cigarette use per day, whereas Figure 2 conveys pack use per day in relation to the equivalent number of cigarettes used daily. These differences may seem small; however, it helps guide users to the right choice when considering that health care providers have been historically trained on social history gathering that emphasizes packs per day and pack-years.
Failure Mode and Effects Analysis
Failure mode and effects analysis (FMEA) is “a structured way to identify and address potential problems, or failures and their resulting effects on the system or process before an adverse event occurs.”9 The benefit of an order set must be weighed against the risk during development. FMEA should be applied during order set design to assess and limit risk just as with any other clinical care process.
FMEA examines both level of risk and frequency of risk occurrence associated with a new proposed process. For example, let’s evaluate an order set designed for pain control after surgery that consists of multiple high-risk opioids along with antihistamine medications for as-needed itch relief (a non-life-threatening adverse event (AE) of opioids well known by the medical community). An interdisciplinary FMEA team consisting of subject matter experts may examine how the process should flow in step-by-step detail and then discuss the benefit of a process and risk for potential error. A FMEA team would then analyze what could go wrong with each part of the process and assign a level of risk and risk frequency for various steps in the process, and then decide that certain steps should be modified or eliminated. Perhaps after FMEA, a facility might conclude that the risk of serious complications is high when you combine opioid use with antihistamine medications. The facility could decide to remove antihistamine medications from an order set if it is determined that risks outweigh benefits. While a root cause analysis might identify the cause of an AE after order set use, these situations can be prevented with FMEA.
When applying FMEA to Figure 1, while bupropion is known as an evidence-based oral tobacco cessation option, there is the possibility that bupropion could be inadvertently prescribed from the order set in a hospitalized patient with alcohol withdrawal and withdrawal seizure history. These potentially dangerous situations can be avoided with FMEA. Thus, although bupropion may be evidence-based for NRT, decisions regarding order set design using EBM alone are insufficient.
The practitioner must consider possible unintended consequences within order sets and target treatment options to the appropriate setting and audience. Although Figure 1 may appear to be more inclusive, the interdisciplinary committee designing the inpatient NRT order set felt there was heightened risk with introducing bupropion in Figure 1 and decided the risk would be lowered by removing bupropion from the redesigned NRT order set (Figure 2). In addition to the goal of balancing availability of NRT options with acceptable risk, Figure 2 also focused on building an NRT order set most applicable to the inpatient setting.
Including Evidence-Based Practices
EBM has become a routine part of clinical decision making. Therefore, including EBM in order set design is vital. EBM for NRT has demonstrated that combination therapy is more effective than is monotherapy to help tobacco users quit. Incremental doses of NRT are recommended for patients who use tobacco more frequently.10
As shown in Figures 1 and 2, both order set designs incorporate EBM for NRT. Although the importance of implementing EBM is evident, critical factors, such as HFE and FMEA make a difference with well-designed order sets.
Results
The 4-step order set design technique was used during development of an inpatient NRT order set at the JAHVH. Results for the inpatient Joint Commission Tobacco Treatment Measures were obtained from the Veterans Health Administration quality metric reporting system known as Strategic Analytics for Improvement and Learning (SAIL). SAIL performance measure outcomes, which include the inpatient Joint Commission Tobacco Treatment Measures, are derived from chart reviews conducted by the External Peer Review Program. Outcomes demonstrated that TOB-2 and TOB-3 (2 inpatient Joint Commission Tobacco Treatment Measures) known as tob20 and tob40, respectively, within SAIL improved by more than 300% after development of an NRT order set using the 4-step order set design framework along with implementation of a multifaceted tobacco cessation care delivery system at JAHVH.
Discussion
While the overall tobacco cessation care delivery system contributed to improved outcomes with the inpatient Joint Commission Tobacco Treatment Measures at JAHVH, the NRT order set was a cornerstone of the design. Although using our order set design technique does not necessarily guarantee successful outcomes, we believe using the 4-step order set design process increases the value of order sets and has potential to improve quality outcomes.
Limitations
Although improved outcomes following implementation of our NRT order set suggest correlation, causation cannot be proven. Also while the NRT order set is believed to have helped tremendously with outcomes, the entire tobacco cessation care delivery system at JAHVH contributed to the results. In addition, the inpatient Joint Commission Tobacco Treatment Measures help improve processes for tobacco cessation care. However, we are uncertain whether the results of our improvement efforts helped patients stop tobacco use. Further studies are needed to determine impact on population health. Finally, our results were based on improvement work done at a single center. Further studies are necessary to see whether results are reproducible.
Conclusion
There was significant improvement with the inpatient Joint Commission Tobacco Treatment Measures outcomes following development of a tobacco cessation care delivery system that included design of an inpatient NRT order set using a 4-step process we developed. This 4-step structure includes emphasis on efficiency and ease of use; human factors engineering; failure mode and effects analysis; and incorporation of evidence-based medicine (Box.) Postimplementation results showed improvement of the inpatient Joint Commission Tobacco Treatment Measures by greater than 3-fold at a single hospital.
The next steps for this initiative include testing the 4-step order set design process in multiple clinical settings to determine the effectiveness of this approach in other areas of clinical care.
1. Order set. http://clinfowiki.org/wiki/index.php/Order_set. Updated October 15, 2015. Accessed August 30, 2019.
2. Bates DW, Kuperman GJ, Wang S, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc. 2003;10(6):523-530.
3. Institute for Safe Medication Practices. Guidelines for standard order sets. https://www.ismp.org/tools/guidelines/standardordersets.pdf. Published January 12, 2010. Accessed August 30, 2019.
4. Heath C, Heath D. Switch: How to Change Things When Change Is Hard. New York, NY: Crown Business; 2010:50-51.
5. US Department of Health and Human Services. Usability testing. https://www.usability.gov/how-to-and-tools/methods/usability-testing.html. Accessed August 30, 2019.
6. World Health Organization. What is human factors and why is it important to patient safety? www.who.int/patientsafety/education/curriculum/who_mc_topic-2.pdf. Accessed August 30, 2019.
7. Sollisch J. The cure for decision fatigue. Wall Street Journal. June 10, 2016. https://www.wsj.com/articles/the-cure-for-decision-fatigue-1465596928. Accessed August 30, 2019.
8. ECRI Institute. Implementing the ENFit initiative for preventing enteral tubing misconnections. https://www.ecri.org/components/HDJournal/Pages/ENFit-for-Preventing-Enteral-Tubing-Misconnections.aspx. Published March 29, 2017. Accessed August 30, 2019.
9. Guidance for performing failure mode and effects analysis with performance improvement projects. https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/QAPI/downloads/GuidanceForFMEA.pdf. Accessed August 30, 2019.
10. Diefanbach LJ, Smith PO, Nashelsky JM, Lindbloom E. What is the most effective nicotine replacement therapy? J Fam Pract. 2003;52(6):492-497.
1. Order set. http://clinfowiki.org/wiki/index.php/Order_set. Updated October 15, 2015. Accessed August 30, 2019.
2. Bates DW, Kuperman GJ, Wang S, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc. 2003;10(6):523-530.
3. Institute for Safe Medication Practices. Guidelines for standard order sets. https://www.ismp.org/tools/guidelines/standardordersets.pdf. Published January 12, 2010. Accessed August 30, 2019.
4. Heath C, Heath D. Switch: How to Change Things When Change Is Hard. New York, NY: Crown Business; 2010:50-51.
5. US Department of Health and Human Services. Usability testing. https://www.usability.gov/how-to-and-tools/methods/usability-testing.html. Accessed August 30, 2019.
6. World Health Organization. What is human factors and why is it important to patient safety? www.who.int/patientsafety/education/curriculum/who_mc_topic-2.pdf. Accessed August 30, 2019.
7. Sollisch J. The cure for decision fatigue. Wall Street Journal. June 10, 2016. https://www.wsj.com/articles/the-cure-for-decision-fatigue-1465596928. Accessed August 30, 2019.
8. ECRI Institute. Implementing the ENFit initiative for preventing enteral tubing misconnections. https://www.ecri.org/components/HDJournal/Pages/ENFit-for-Preventing-Enteral-Tubing-Misconnections.aspx. Published March 29, 2017. Accessed August 30, 2019.
9. Guidance for performing failure mode and effects analysis with performance improvement projects. https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/QAPI/downloads/GuidanceForFMEA.pdf. Accessed August 30, 2019.
10. Diefanbach LJ, Smith PO, Nashelsky JM, Lindbloom E. What is the most effective nicotine replacement therapy? J Fam Pract. 2003;52(6):492-497.