Unexpected Drawbacks of Electronic Order Sets | AHRQ Patient Safety Network
WebM&M logo

Unexpected Drawbacks of Electronic Order Sets
A transition from paper orders to CPOE left out an important safety reminder, resulting in mismanagement of an elderly patient's low potassium and magnesium levels. This led to a fatal arrhythmia. The paper-based electrolyte order set had provided a reminder that magnesium replacement should accompany potassium replacement; however, in the computerized system, a separate order set was necessary for each electrolyte. In the accompanying commentary, John D. McGreevey III, MD, of the University of Pennsylvania, discusses the benefits and limitations of order sets and the importance of clinicians maintaining a trust-but-verify approach to decision support systems.

Cases & Commentaries
Published November 2016

Unexpected Drawbacks of Electronic Order Sets

Commentary by John D. McGreevey III, MD

The Case

A 70-year-old man with stage 4 prostate cancer presented to the emergency department with severe muscle weakness. His lab results revealed severe hypokalemia and hypomagnesemia (2.2 mmol/L and 1.2 mEq/L, respectively). His electrocardiogram was significant for QTc prolongation and U-waves. The admitting team noted the patient had recently been started on abiraterone, an antiandrogen drug known to cause hypokalemia.

Because the patient had symptomatic hypokalemia, the medical resident was instructed to order potassium replacement using the hospital order set. Using the patient's potassium value to select the appropriate hypokalemia order set, the resident signed the order in the new electronic health record. However, while receiving his IV potassium repletion, the patient developed ventricular tachycardia and became nonresponsive. No resuscitation efforts were made due to the patient's DNR status.

Hypomagnesemia often accompanies hypokalemia. Despite the patient's low serum magnesium levels, neither the resident nor medical team recognized that the patient required magnesium replacement along with the potassium. Before the hospital transitioned to computerized provider order entry (CPOE), orders for common electrolytes were available on a single paper order set with check-boxes to indicate the appropriate replacement dose and route for specific lab values, including potassium and magnesium.

When the hospital recently switched from paper orders to CPOE, all order sets were imported into the electronic health record. However, to minimize the amount of information presented on each screen, a separate order set was created for each electrolyte replacement order (e.g., hypokalemia, hypomagnesemia). The team responsible for transitioning the paper orders to CPOE did not realize that the old paper electrolyte order set provided a reminder to order magnesium replacement to accompany potassium replacement, and that, by separating out the order sets electronically, an important safety reminder was lost. An internal investigation concluded that the transition to electronic order sets contributed to mismanagement of the patient's low magnesium and potassium levels, which resulted in a fatal arrhythmia.

The Commentary

by John D. McGreevey III, MD

Order sets are intended to increase consistent, evidence-based practice, prevent errors of omission, provide clinical decision support, and enhance providers' ordering efficiency. Principles of order set development, design, and governance have been described extensively.(1-9) However no guidelines exist regarding types of order sets that should always be created. In practice, electrolyte repletion order sets are the fifth most commonly used task-oriented order set.(5) As in this case, multiple electrolyte abnormalities may exist, and it can be clinically important to address these problems concurrently. Yet the optimal way to organize and present electrolyte repletion orders within electronic health records (EHRs) is unknown.

One potential way to promote repletion of multiple electrolytes within an EHR is through corollary orders. Corollary orders are prompted by a triggering order and are typically presented to promote the safe use of a drug or therapy.(10) For example, an order for amphotericin B (triggering order) might prompt the presentation of orders for potassium and magnesium levels to the ordering provider, as these electrolytes should be monitored during amphotericin B therapy. Corollary orders have been shown to decrease provider errors of omission and could be designed such that an order for one electrolyte (in this case, potassium) triggers a corollary order for another electrolyte (here, magnesium).(10) The advantage of a corollary order approach is that it reminds providers to consider repletion of electrolytes that tend to require corepletion. The disadvantage is that unnecessary repletion of corollary order electrolytes could occur, as not every instance of hypokalemia is accompanied by hypomagnesemia.

An alternative to corollary orders for the repletion of multiple electrolytes is to group electrolyte orders together within a single order set. An order set containing multiple electrolyte repletion orders can be on paper, as in this case, or within the EHR. In this case, the medical team may have assumed that magnesium repletion guidance would be offered automatically upon ordering potassium repletion within the EHR, since they were accustomed to magnesium and potassium guidance being linked on the prior paper order set. However, no reminder was forthcoming, and the physicians neglected to replete magnesium—an error that proved fatal in this instance.

Simplification is one of the tenets of well-designed clinical decision support.(6,9) In this case, we are told that developers decided to create a separate order set for each electrolyte in order to simplify the ordering screen. However, the goals of providing robust decision support and simplicity should not be mutually exclusive. When using paper order sets, presenting related orders on the same page essentially serves as decision support; the clinician has an immediate visual reminder to consider ordering magnesium when repleting potassium. In contrast, within an EHR, there may be ways to harness the functionality of the system to guide appropriate ordering without placing all orders on the same screen. As an example, some EHRs can proactively monitor patient data, such as lab results (e.g., potassium), problems on the problem list (e.g., community-acquired pneumonia), and suggest appropriate orders or order sets as a result.(3)

At our institution—like that in the case—potassium, magnesium, and other electrolytes are ordered separately within the EHR. Although concomitant hypokalemia and hypomagnesemia frequently occur and often require corepletion, some circumstances do not require repletion of both electrolytes. As an example, in patients with congestive heart failure and chronic kidney disease taking diuretic therapy, it may be clinically appropriate to replete potassium as necessary, but not magnesium. On the other hand, we have chosen to couple our warfarin order with an order for prothrombin time/international normalized ratio monitoring within the warfarin order set. These two decisions illustrate that the choice to link two related orders or to keep them separate is not always straightforward.

Clinicians and, if possible, clinical informaticists should participate in the transition process from paper to electronic order sets to ensure that new order sets are consistent with evidence-based practice and provide necessary decision support.(3,11) Clinician involvement could have been helpful in this case to recognize the decision support provided in the paper order set and to advocate for such decision support in the electronic order set. Sometimes, detailed conversations with the EHR vendor may be needed to identify the best way to preserve, and even enhance, decision support in the EHR. Ultimately, the failure to recognize and maintain decision support that had been provided in the paper order set was a contributing factor to the outcome in this case.

There are limited data regarding the unintended consequences of order sets. Many order sets are underutilized and a relatively small proportion of order sets account for the vast majority of use, suggesting that there are opportunities to be more judicious with order set development.(1,5) Order sets can suffer from lack of standardization, usability violations, inadequate governance, insufficient stakeholder involvement, and failure to follow best practices for medication ordering.(3,11-13) Allowing individually developed ("personalized") order sets can lead to undesirable variations in care.(3,5,8,14) Because order sets facilitate the ordering process, they may encourage overordering, such as with sedating medications.(15)

Lastly, a fundamental question raised by this case is whether providers expect clinical decision support to serve as a failsafe against errors in clinical judgment. One of the ironies of the rapid adoption of EHR technology in the United States is that clinicians can develop excessive reliance on order sets and other tools within the EHR for clinical decision-making, a risk that has been reported in other settings that experience increasing automation as well.(16,17) Well-designed EHRs with clinical decision support can enhance patient care. However, EHR implementations have also been associated with new types of errors. Some of these errors can be difficult for end-users to recognize, and CPOE may fail to protect against them.(16,18-20) Despite best efforts of developers to build sufficient decision supports within EHRs, it is impossible for technology to compensate for every potential omission in clinical care plans. As such, providers should adopt and maintain a "trust but verify" approach when interacting with EHR systems. The tension inherent in order sets is that they can remind clinicians about appropriate steps in care, but they cannot replace the need for providers to make those decisions independently based on medical knowledge, experience, and clinical judgment.

Take-Home Points

Order sets can provide clinical decision support, promote consistent evidence-based practice, decrease errors of omission, and support provider efficiency. However, they also have limitations.
Adherence to established principles of order set design, decision support, and governance is critical to the creation and maintenance of successful order sets.
The optimal design of electrolyte repletion orders and order sets is not entirely clear and may vary from institution to institution.
When transitioning from paper to electronic order sets, clinicians and others involved in order set development should identify decision support in the former and create this support in the latter—it may involve a different mechanism than when on paper.
Providers should be mindful of developing excessive trust and reliance on CPOE systems—these tools should not be viewed as a replacement for clinical judgment.

John D. McGreevey III, MD
Assistant Professor of Clinical Medicine
Perelman School of Medicine at the University of Pennsylvania
Associate CMIO, University of Pennsylvania Health System
Philadelphia, PA

References

1. Payne TH, Hoey PJ, Nichol P, Lovis C. Preparation and use of preconstructed orders, order sets, and order menus in a computerized provider order entry system. J Am Med Inform Assoc. 2003;10:322-329. [go to PubMed]

2. Chisolm DJ, McAlearney AS, Veneris S, Fisher D, Holtzlander M, McCoy KS. The role of computerized order sets in pediatric inpatient asthma treatment. Pediatr Allergy Immunol. 2006;17:199-206. [go to PubMed]

3. Bobb AM, Payne TH, Gross PA. Viewpoint: controversies surrounding use of order sets for clinical decision support in computerized provider order entry. J Am Med Inform Assoc. 2007;14:41-47. [go to PubMed]

4. ISMP's Guidelines for Standard Order Sets. Horsham, PA: Institute for Safe Medication Practices; March 2010. [Available at]

5. Wright A, Feblowitz JC, Pang JE, et al. Use of order sets in inpatient computerized provider order entry systems: a comparative analysis of usage patterns at seven sites. Int J Med Inform. 2012;81:733-745. [go to PubMed]

6. McGreevey JD III. Order sets in electronic health records: principles of good practice. Chest. 2013;143:228-235. [go to PubMed]

7. Leu MG, Morelli SA, Chung OY, Radford S. Systematic update of computerized physician order entry order sets to improve quality of care: a case study. Pediatrics. 2013;131(suppl 1):S60-S67. [go to PubMed]

8. SAFER Guides: Computerized Provider Order Entry With Decision Support. Washington, DC; January 2014. [Available at]

9. Bates DW, Kuperman GJ, Wang S, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc. 2003;10:523-530. [go to PubMed]

10. Overhage JM, Tierney WM, Zhou XH, McDonald CJ. A randomized trial of "corollary orders" to prevent errors of omission. J Am Med Inform Assoc. 1997;4:364-375. [go to PubMed]

11. Wright A, Sittig DF, Ash JS, et al. Governance for clinical decision support: case studies and recommended practices from leading institutions. J Am Med Inform Assoc. 2011;18:187-194. [go to PubMed]

12. ISMP Medication Safety Alert! Acute Care Edition. ISMP develops guidelines for standard order sets. March 11, 2010. [Available at]

13. Khajouei R, Peek N, Wierenga PC, Kersten MJ, Jaspers MW. Effect of predefined order sets and usability problems on efficiency of computerized medication ordering. Int J Med Inform. 2010;79:690-698. [go to PubMed]

14. Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc. 2006;13:547-556. [go to PubMed]

15. Bates DW, Leape LL, Cullen DJ et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA. 1998;280:1311-1316. [go to PubMed]

16. Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The extent and importance of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc. 2007;14:415-423. [go to PubMed]

17. Bainbridge L. Ironies of automation. Automatica. 1983;19:775-779. [Available at]

18. Metzger J, Welebob E, Bates DW, Lipsitz S, Classen DC. Mixed results in the safety performance of computerized physician order entry. Health Aff (Millwood). 2010;29:655-663. [go to PubMed]

19. Schiff GD, Amato MG, Eguale T, et al. Computerised physician order entry–related medication errors: analysis of reported errors and vulnerability testing of current systems. BMJ Qual Saf. 2015;24:264-271. [go to PubMed]

20. Koppel R, Metlay JP, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA. 2005;293:1197-1203. [go to PubMed]