Emergency Severity Index (ESI): A Triage Tool for Emergency Department
Chapter 8. Evaluation and Quality Improvement
To maintain reliability of the Emergency Severity Index (ESI) in an individual institution, it is important to evaluate how the system is being used. A natural learning curve will occur and it can be easy for nurses to fall back into maladaptive triage habits, or become concerned about triaging "too many level-2 patients" when the waiting room is crowded. Additionally, new models of triage intake are being used by EDs across the United States. Physicians, nurse practitioners, and physician assistants may play a role at triage. It is important that anyone who performs the triage assessment and is responsible for assigning a triage level upon presentation be competent in ESI. Continuous evaluation using standard quality improvement (QI) methods will help ensure that reliability and validity of the system is maintained by all.
In 2001, the IOM published the report, "Crossing the Quality Chasm, A New Health System for the 21st Century," which defined quality healthcare and identified six aims to improve the overall quality of healthcare (Institute of Medicine, 2001). The IOM defined quality healthcare as, "The degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge." The six aims of quality include improving the safety, effectiveness, patient-centeredness, timeliness, efficiency and equity of the healthcare system and are defined in Table 8-1 (Institute of Medicine, 2001). The triage process is probably one of the highest risk areas in the ED and attention to quality monitoring is important. All six aims can be used to evaluate the triage process. Emergency departments can structure their quality improvement (QI) monitoring process around any or all of the six IOM aims. Specific examples will be discussed below.
Table 8-1 The Six Institute of Medicine Aims
| IOM Aim | Definition |
|---|---|
| Safety | Avoiding injuries from care that is intended to help |
| Effectiveness | Providing services based on evidence and avoiding interventions not likely to benefit |
| Patient-Centeredness | Respectful and responsive to individual patient preferences, needs, values, in clinical decision making |
| Timeliness | Reducing waits and sometimes harmful delays for those who receive care |
| Efficiency | Avoiding waste, in particular of equipment, supplies, ideas, energy |
| Equity | Care that does not vary in quality due to personal characteristics (gender, ethnicity, geographic location, or socio-economic status) |
It is also important to choose a system by which the improvement can be readily assessed. When choosing a method to evaluate the success or failure of implementation, it is important to remember why the triage process was changed. The following reasons are frequently identified as driving forces to change existing triage processes:
- Reduction in variation of assigned triage categories and the ability for everyone to "speak the same language" regarding triage categories.
- Decreased risk of negative outcomes due to mistriage, particularly while patients are waiting.
- The ability to obtain more accurate data to use for administrative purposes.
- The need to move from a three-category to a five-category triage system to better "sort" the increasing number of ED patients.
- A more accurate description of patient triage levels and departmental case mix (Wuerz, Milne, Eitel, Travers, & Gilboy, 2000).
The ultimate goal of ESI implementation is to accurately capture patient acuity to optimize the safety of patients in the waiting room by ensuring that only patients stable to wait are selected to wait. Patients should be "triaged" according to acuity of illness. When a reliable and valid triage system such as ESI is used, the triage score can then be used as administrative data to accurately describe departmental case mix, beyond admission and discharge status. With this in mind, it is important that every patient be assigned a triage score on arrival. The primary goal of conducting QI activities of the ESI triage system is to maintain reliability and validity of the system implementation. If triage nurses are not assigning scores accurately, then the data cannot be used for any purpose, either real time or for other administrative purposes. With the addition of new nurses or other providers at triage, and natural trends over time, it is important to, at a minimum, always monitor the accuracy of the triage level. It is also important to clearly articulate to the ED staff what is not a goal of ESI triage implementation. For example, ESI triage alone cannot decrease the ED length of stay or improve customer satisfaction with the ED visit.
ESI Triage Quality Indicators and Thresholds
In any QI plan, it is important to select meaningful indicators to monitor. Donabedian's trilogy of structure, process, and outcome (Table 8.2) can be used to select the type of indicator (Donabedian, 1992). All indicators can be organized around Donabedian's structure and the six IOM aims of improving quality care: safety, effectiveness, patient-centeredness, timeliness, efficiency, and equity. Selected examples are included in Tables 8.2 and 8.3. The tables include indicators specific to monitoring implementation of ESI and suggest other indicators which can be used to evaluate other aspects of the broader triage process.
Table 8-2. Donabedian's Trilogy
| Definition (Example) | |
|---|---|
| Structure | How care is organized (standing protocols which allow nurses to give acetaminophen for fever at triage) |
| Process | What is done by caregivers (proportion of patients with fever at triage who receive acetaminophen at triage) |
| Outcome | Results achieved (fever reduction within one hour after arrival) |
While selecting QIs to review is critical, it is also important to recognize specific indicators that are not appropriate to review. For example, the actual number of resources that were used in providing care to the patient is NOT an appropriate quality indicator to monitor. Resources are incorporated in the ESI algorithm only to help the triage nurse to differentiate among the large proportion of patients that are not critically ill. Monitoring the number of resources used "on the back end" may further increase the triage nurses' focus on counting resources, which is not the most important component of the algorithm. However, knowledge of the standard of care will serve to increase accuracy in assessing the resources used for various presentations, allowing for accurate triage assignment.
In addition to selecting useful QIs, it is also important that the ED management team select a realistic threshold to meet for each indicator. All indicators do not need to have the same threshold. For example, when reviewing accuracy of triage categorization, a realistic goal must be determined. Should the triage category be correct 100 percent, 90 percent, or 80 percent of the time? Frequently a threshold of 90 percent is selected. However, the goals and circumstances of each department may be unique and should be considered when selecting each indicator and threshold. For example, the ED management team might stipulate that, when in doubt about a patient's triage rating, nurses err on the side of over-triage. While this approach might result in some patients being mistriaged as more acute than they actually are, it is preferable to risking an adverse event because the patient was triaged to a less urgent category. In this ED, the triage accuracy threshold might be 80 percent, with a goal to keep the mistriage rate at 20 percent.
Finally, it is also important to determine how many triage indicators should be monitored on an ongoing basis. It is reasonable to select one or more indicators. The number of indicators to be monitored will be determined by available staff resources and the relationship of ESI indicators to other QIs that are routinely monitored. It is also possible to focus on monitoring one aspect of triage for a period of time and then switch to another indicator when improvement occurs in the previously monitored indicator. Various levels of indicators could also be measured, e.g., shift-level or a day-of-week level of evaluation.
Accuracy of triage acuity level should probably be monitored on a continuous basis to evaluate new triage nurses as well as monitor for trends which may identify the need for re-education on a particular aspect of triage. These data can be reported as the proportion of correctly assigned triage levels. In addition, a more formal evaluation of inter-rater reliability can be periodically conducted by having a proportion (example, 20%) of randomly selected nurses from all shifts assign triage levels to pre-selected paper cases It is recommended that at least 10 paper cases are used for this type of evaluation. This evaluation will measure how often the triage nurses in an individual department would assign triage levels the same, or would "agree". This can be a valuable exercise to conduct on a regular basis (e.g. after key changes in departmental processes) if resources are available. It may also be appropriate to evaluate triage acuity accuracy more often in a department with higher nurse turnover.
ESI Triage Data Collection
The method of collecting QI data for ESI triage indicators can be incorporated into the data collection process for other ED QIs or data can be collected as a separate process. The method of data collection will depend on the indicator selected, the availability of triage experts, and logistic issues such as accessibility to electronic versus paper ED records. For example, if "accuracy of triage category" is selected as a triage QI, a triage expert is needed to review the triage categories.
Accuracy of the triage category assigned is a critical indicator and should be monitored when ESI is first implemented. If it is determined that the institution wishes to measure ED length of stay or wait times to see the physician for each ESI triage category, it is preferable to have access to electronic information to successfully monitor this indicator. Without electronic sources, these data are cumbersome to track and manual calculations most likely result in error. It is also advisable to monitor medians instead of means when evaluating any indicator that is a time measure (e.g. time to physician assessment). When calculating means, the resultant values are typically very skewed, and are therefore not an accurate measure. Standardized time interval nomenclature is starting to appear in the literature however, it is important to reiterate that ESI does not stipulate times to care. Finally, when monitoring QIs, it is important to determine how many charts must be reviewed for each indicator and how frequently the indicator should be reviewed (monthly, quarterly, etc.). Selection of the appropriate number of charts for each indicator will depend on the particular indicator. If wait times for each category are reviewed, data will be most accurate when a large percentage of cases, preferably all, are reviewed.
Routine evaluation of the accuracy of ESI should reflect an appropriate number of randomly selected charts. Cases from different nurses and each shift and day of the week should be reviewed. Ten percent of all cases is often selected as an appropriate number of cases to review. In a busy ED, this is often an unrealistic number. It is important for each institution to consider the number of review staff, their backgrounds, and their availability. It is also prudent to evaluate ESI rating accuracy for individual cases where there was a near-miss or an adverse event related to the triage process.
When determining the frequency of triage audits, the institution should consider other departmental QI activities and try to integrate the review of triage indicators into the same process and schedule.
It is also very helpful to involve the triage nurses in data collection. Peer reviews are a useful way to raise awareness about triage accuracy.
Sharing Results and Making Improvements
Often, 95 percent of the time and attention to QI and process improvement activities is given to the monitoring stage of the process, and little attention is paid to evaluating the data and determining process improvements. The "numbers"" are often posted somewhere and little is done to actually improve the outcomes. The most important component of QI is sharing the data and discussing ways to improve the results. Positive systems outcome in triage improvement depends on measuring, analyzing data, and then educating the staff. All staff should be aware of the triage QI, the current overall incidence in which the threshold is met, and the actual goal. For example, if the accuracy of the triage category is being monitored and continues to be reported as 60 percent, intervention is necessary.
Examples of ESI Triage Indicators
The emergency departments described below have implemented ESI and a QI program. They have provided examples of how they incorporate triage indicators into their QI plan.
Hospital 1. At hospital 1, the accuracy of triage nurses' ESI triage ratings is assessed on a continuous basis and reported quarterly as one indicator of the overall ED QI plan. This indicator has been monitored since ESI was implemented and continues to be the only triage indicator monitored to date. Each week, three different nurses randomly select five charts to review with the ED clinical nurse specialist (CNS). The assessment team reviews many different general documentation indicators, including the accuracy of the ESI triage category. The CNS is the designated triage expert and discusses each case with the staff nurse as she reviews the records. When there is a disagreement, cases are reported as mistriages for the QI report. The assessment team collects and retypes all mistriages as an educational tool that includes an explanation of the correct triage category. These cases are compiled in a handout and distributed to all staff nurses monthly. The assessment team reviews sixty charts monthly.
Hospital 1 has noted several distinct advantages of the triage accuracy review:
- All ED staff nurses are aware of the QI indicators; case examples provide individual nurses with the opportunity to reflect on their own practice with similar case scenarios. Staff nurses have the opportunity to discuss each case with the CNS to obtain additional insight.
- All nurses benefit from the discussion when the cases are distributed as a teaching tool.
Hospital 1, like many other EDs, also has excellent information technology resources that facilitate quality monitoring of clinical information. The triage acuity is part of the electronic medical record. It is possible to track time to physician evaluation for each triage category. This can be powerful administrative data. This data is far more powerful when describing overall ED acuity than using hospital admission data to describe overall ED acuity.
Hospital 2. At hospital 2, several triage indicators are reviewed on a regular basis. The ESI rating assigned by the nurse at triage and time data are recorded in the hospital's computer information system during the ED visit. The electronic information is compiled for monthly QI monitoring. Time data are reported by ESI triage level, including the following:
- Total ED length of stay.
- Time from triage to placement in the ED bed.
- Time from triage to being seen by the ED physician.
- Time from placement in the ED bed to discharge.
The time data are used for many purposes, such as monitoring for operational problems that lead to increased length of stay. The time data prove useful in addressing issues related to specific patient populations at hospital 2's ED. For example, the time data were tracked for psychiatric patients and subsequently a new policy regarding psychiatric consults was developed. The policy stipulates response times for the crisis team to see ED psychiatry patients and is based on ESI triage level. Information about the number of patients triaged to the various areas of the ED (medical urgent care, minor trauma, pediatrics, acute) is also reported by ESI triage level on a monthly basis. These data are used to make operational decisions, such as the time of day that medical urgent care and minor trauma services are offered.
The accuracy of triage nurses' ESI ratings is reviewed as part of the QI program at hospital 2. The initial review was conducted during the first few months after implementation of the ESI. The nurse educator reviews a random sample of ED charts on a regular basis to assess the accuracy of the triage nurses' ESI ratings. Individual nurses receive feedback and trends are reported to the entire nursing staff.
Accuracy of triage ratings are also reviewed as an indicator at hospital 2, through a monthly peer chart review process. Each nurse selects two random ED charts per month and reviews many aspects of nurses' documentation, including the ESI triage rating. The review is forwarded to nursing leadership for followup with individual nurses. Any important trend identified is communicated to the entire staff.
Another QI effort at hospital 2 is the review of all ESI level-3 patients triaged to the medical urgent care (fast track) area. The nurse manager receives a monthly report, compiled with electronic data from the hospital computer system, of all ESI level-3 patients triaged to medical urgent care and all ESI level-4 and level-5 patients triaged to the ED. Though the department has a guideline that ESI level-4 and level-5 adult patients are primarily triaged to medical urgent care or minor trauma, and ESI levels-1, -2, and -3 adult patients are primarily triaged to the acute ED, the triage nurse is allowed discretion in triaging these patients. The ongoing review of the ESI level-3 patients sent to medical urgent care allows the leadership team to review the accuracy of the nurses' triage decisions.
Hospital 3. At hospital 3 the manager assigns experts to review triage categories. The manager and clinical coordinators review charts identified by peers as potential mistriages. The expert group reviews the chart and discusses it with the triage nurse. The team of experts spot check charts frequently. If a trend is noticed, the expert group will post the case so that all staff can learn from it.
Hospital 4. At hospital 4, the manager created a log after initiation of the ESI triage system. The triage nurse logged the patient name, triage nurse name, triage level and rationale and resources for each patient triaged. The management team reviewed each chart for triage category accuracy either while the patient was in the department or the next day. The management did this for the first 2 weeks and again in 3 months. The purpose of this monitoring activity was to assess the triage nurses' understanding of resource definitions.
Hospital 5. Hospital 5's strategic plan called for the hospital to increase the number of trauma and stroke patients they would accept from outlying hospitals. Most of these patients were emergency-department-to-emergency-department transfers. Many of these patients arrived intubated and others were intubated on arrival. The staff felt that the acuity of the ED patient population was rising quickly. Nursing leadership chose to look at case mix data (the number of patients in each ESI category) for 1 year and was able to make adjustments to staffing to cover increases in patient acuity. This is an excellent example of why it is so important that every patient, including traumas and cardiac arrest patients, receive an accurate triage category. This allows each hospital to benchmark their case mix data with other institutions. These patients are clearly not "triaged" at "triage," but they represent an important group of patients seen in the ED. If the primary nurse does not assign a triage category to these patients, the ED case mix data will significantly under-represent the higher acuity of the department.
It is important for the emergency department nursing leadership to implement a QI plan. The plan should generate meaningful data that can be shared with the ED staff on a regular basis. Issues with individual triage nurses must be promptly identified and education provided. Larger trends also must be identified rapidly and accompanied by an appropriate response including communication with senior level leadership to plan for change. The members of the ESI research team are repeatedly asked about quality assurance and our suggestion is to keep it simple, relevant, and meaningful.
Note: Appendix A of this handbook includes frequently asked questions and post-test assessment questions for Chapters 2 through 8. These sections can be incorporated into a locally-developed ESI training course.
References
Donabedian A (1992). Quality assurance: Structure, process and outcomes. Nursing Standard 11(Suppl QA):4-5.
Institute of Medicine (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academies Press.
Page last reviewed October 2014
Internet Citation: Chapter 8. Evaluation and Quality Improvement. October 2014. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/professionals/systems/hospital/esi/esi8.html
No hay comentarios:
Publicar un comentario