The association between urgency level and hospital admission, mortality and resource utilization in three emergency department triage systems: an observational multicenter study

Abstract

Background

Effective triage systems are crucial for prioritizing patients based on urgency and optimizing resource utilization. An ideal triage system is expected to have low resource utilization, hospitalization and mortality among patients classified at low urgency levels. Furthermore, it should exhibit an increase in the risk of hospitalization and mortality as urgency levels increase, ensuring the most critically ill patients receive priority care first. However, it is unclear which triage system performs best.

Objective

To compare the performance of the Manchester Triage System (MTS), the Emergency Severity Index (ESI), and the Netherlands Triage Standard (NTS) by investigating the association between urgency levels and resource utilization, hospitalization and in-hospital mortality in Emergency Department (ED) patients.

Methods

Observational multicenter cohort study using data from the Netherlands Emergency department Evaluation Database, comprising seven representative EDs in six Dutch hospitals. All consecutive ED patients with a registered urgency level were included. Resource utilization, hospitalization and mortality were measured across all urgency levels. In each triage system, multivariable logistic regression was used to assess the association between urgency level and in-hospital mortality and hospitalization, adjusting for age, sex, presenting complaints and hospital type.

Results

A total of 696,518 ED visits (MTS 320,406 (46.1%), ESI 214,267 (30.8%), NTS 161,845 (23.3%) patients) were included. Resource utilization was substantially lower in the lowest urgency level of the ESI compared to the MTS and NTS. Hospitalization to a regular ward, cardiac, medium or intensive care unit in the least urgent level was 3.9% in the ESI, considerably lower than in the MTS (23.1%) and NTS (34.3%) (P < 0.05). Mortality in the lowest urgency level of the ESI was 0.8%, while in the MTS and NTS this was 6.3% and 12.4%, respectively (P < 0.05). In the ESI, the risk (Adjusted Odds Ratios) for hospitalization and mortality increased much more with increasing urgency levels compared to the MTS and NTS.

Conclusion

This study suggests that the ESI may be more effective in distinguishing between patients with low and high urgency, with a reduced risk of undertriage when compared to the MTS and NTS.

Introduction

Emergency department (ED) overcrowding is a global problem and a threat for patient safety, as well as the satisfaction of patients and healthcare providers [1,2,3]. There is a high and persistently increasing demand for emergency care while limited capacities and resources are available [1,2,3,4,5,6]. This higher demand on EDs may lead to poorer patient outcomes and quality of care [1,2,3, 6]. Adequate prioritization of ED patients using a triage system, in which patients are treated based on their urgency level and expected resources, is important and may increase patient safety [2, 3, 7, 8]. By categorizing patients into five or six different urgency levels, allowing patients with lower urgency to wait and ensuring more efficient allocation of time and resources to patients presenting with a higher urgency [8,9,10,11].

Despite the widespread use of various triage systems in clinical practice globally, multiple studies have demonstrated significant variability in the predictive performance of commonly implemented triage systems, such as the Emergency Severity Index (ESI) and the Manchester Triage System (MTS) [7, 8]. Additionally, several triage systems have been associated with considerable mistriage, either underestimating or overestimating the urgency of a patient’s condition, i.e. under- or overtriage [7, 8, 12]. It is unclear which triage system performs best in terms of predictive performance and mistriage due to a variety of outcome measures and study designs. More importantly, previous studies were often limited by a single center design or analysis of a single triage system, using small sample sizes, limited data or studying a specific population [7, 8, 13,14,15].

Currently, different triage systems are used in the Netherlands: The MTS, the ESI, the Netherlands Triage Standard (NTS) and local hospital triage tools [16]. Despite the usage of the NTS at several EDs in the Netherlands, little is known about the reliability, validity and performance of the NTS [17,18,19,20].

Comparison of the association between urgency levels and clinical outcomes in different triage systems will provide insight in which triage system is least affected by mistriage and has the best predictive performance. This information can be used for the improvement of current guidelines. Ideally, in a well performing triage system, a limited number of hospitalizations and no deaths would be expected in patients triaged as non-urgent. Furthermore, in the lower urgency levels, the number of used resources should be low [21]. Conversely, one expects the highest hospital admission and mortality rate in the highest urgency levels. Finally, there should be a clear association between the urgency level and the resources used, as well as mortality and hospitalization rate, independent of patient characteristics like age, sex and presenting complaints.

Given the variability in the performance of triage systems, a direct comparison of their ability to predict hospital admission, in-hospital mortality, and resource utilization is essential. Therefore, this study aims to investigate the association between urgency levels and hospital admission, in-hospital mortality and resource utilization in ED patients among three regularly used triage systems.

Methods

Study design and setting

An observational multicenter cohort study was conducted using data from the Netherlands Emergency department Evaluation Database (NEED). The NEED is a quality registry of EDs in the Netherlands providing insight in the quality of national ED care (www.stichting-need.nl). A detailed description of the data collected within the NEED is available in previously published studies utilizing this database [22, 23]. During the study period, the NEED existed of seven EDs in the Netherlands in six hospitals; two academic and four general hospitals [24]. The Medical Ethics Review Committee at Radboudumc determined that this study was exempt from the Medical Research Act and waived the need for informed consent (file no. 2023–16756).

Study population

All consecutive ED patients with a registered urgency level were included in this study.

Definitions

MTS: Manchester triage system

The MTS is a five-level ED triage system which assigns an urgency level based on the patient’s signs and symptoms. It consists of a reductive system using 53 different flowcharts [10]. Ruling out high priority signs or symptoms is necessary to reduce the patient’s urgency level. The urgency levels are divided into red (immediate), orange (very urgent), yellow (urgent), green (standard), blue (non-urgent) [10].

ESI: emergency severity index

The ESI is an ED triage system assigning priority using an algorithm based on the patient’s stability, vital signs and expected resources, resulting in an urgency level from one (most urgent) to five (least urgent) [9].

NTS: Netherlands triage standard

The NTS is a Dutch triage standard used by EDs, general practice centers and ambulance control rooms. It is a six-level system describing urgency levels from U0 (resuscitation) to U5 (no risk of harm, next workday) based on the patient’s condition [11].

Data collection

Data from six hospitals was collected between January 1st, 2017 and December 31st, 2022. Hospital characteristics are displayed in Table 1. Resource utilization data, including blood tests, radiology, electrocardiogram (ECG) and interventions, were automatically extracted from hospital information systems via an Application Programming Interface (IPA) and transferred to the NEED. For example, any recorded blood test (i.e. hemoglobin, troponin, leukocytes) was registered as a resource utilization event.

Table 1 Hospital characteristics

Participants within the different triage systems were stratified into four urgency levels: ‘not urgent’, ‘urgent’, ‘very urgent’ and ‘most urgent’. The NTS originally comprises six levels, wherein the lowest urgency levels of the NTS (‘negligible risk of harm, within 24 hours’ and ‘no risk of harm, next workday‘) were already combined in the database into a single category labelled ‘can be seen next day’. In addition, in the current database the two lowest urgency levels across all systems were merged into the ‘not urgent’ category, as the lowest level (‘can be seen next day’) included a small group of patients. Presenting complaints according to MTS, ESI and NTS were merged into one variable of synchronized presenting complaints (Additional File 1).

Outcome measures

The primary outcomes were in-hospital mortality, hospital admission and resource utilization. In-hospital mortality was defined as death during hospital admission. Patients who died at the ED were not defined as in-hospital deaths but considered a separate category. Patients who died before arrival at the ED were excluded from analyses. Hospitalization included admission to a regular ward, medium care unit (MCU), intensive care unit (ICU), cardiac care unit (CCU) or transfer to another hospital. Resource utilization included the use of diagnostic tests, consultations and provided treatment. Secondary outcomes were length of ED stay, length of hospital stay and ED-revisit.

Statistical analysis

Sample size calculation

Using the rule of thumb, at least ten events per potential confounder are necessary to avoid overfitting. To adjust for the thirteen variables, a minimum of 130 patients who died or were hospitalized was required. The NEED exceeded this threshold.

Descriptive statistics

Skewed data were presented as median with interquartile range. Categorical data were presented as number with percentages.

Main statistical analysis

Multivariable binary logistic regression analysis was used to assess the association between urgency level and in-hospital mortality and hospital admission in the three different triage systems. The primary association of interest was adjusted for age, sex, top ten presenting complaints and hospital type (academic versus general hospital). In all analyses, ‘not urgent’ was defined as the reference group. The reference category for hospital type was ‘general’ and ‘other’ for the top-10 presenting complaints. Patients who died before or upon arrival at the ED were excluded from the regression analysis.

Robustness was tested by adding and removing covariates, thereby ensuring the consistency on the association between triage category and outcomes. Multicollinearity was assessed by examining Variance Inflation Factors (VIFs). Adjusted odds ratios (AORs) were reported with 95% confidence intervals. A P-value < 0.05 was considered to be statistically significant.

Patients with missing data were excluded from the analyses. Data were analyzed using IBM SPSS Statistics (version: 29.0.0.0.).

Results

Patient inclusion and characteristics

The patient flowchart is shown in Additional File 2. A total of 696,518 ED visits were included for analysis. Patient characteristics are presented in Table 2, with additional characteristics presented in Additional File 3. The total MTS cohort comprised 320,406 (46.1%) patients, the ESI 214,267 (30.8%) patients and the NTS 161,845 (23.3%) patients. The median age of the cohort was 56 years old, with a majority of the patients being male (52.0%). The median age increased with higher urgency levels across all triage systems. The proportion of patients classified with the lowest urgency level was relatively low in the ESI cohort (12.9%), compared to the MTS (29.2%) and NTS (39.0%) cohorts. The percentage of patients marked with the highest urgency level was significantly higher in the NTS group (5.4%) than in the MTS (1.5%) and ESI (1.0%) groups.

Table 2 Patient characteristics in different triage systems and urgency levels

Approximately one-third of the patients arrived at the ED by ambulance, with most being referred by the general practitioner. Notably, a significant proportion of patients in the ‘not urgent’ group of the NTS were referred by a hospital specialist (51.6%). In contrast, the ‘most urgent’ group of the NTS had a relatively higher percentage of referrals from general practitioners (3.5%) compared to the MTS (0.4%) and ESI (0.3%).

Resource utilization

Resource utilization and time spent in the ED is displayed in Table 3. Relative resource utilization, defined as resource use within a specific urgency level, divided by the total number of patients in that urgency level for the given triage system, is presented in Additional File 4. In the lowest urgency level of the ESI the fewest resources were utilized compared to the other two triage systems. In contrast, overall, the NTS had the highest resource utilization in both the lowest and highest urgency levels, while showing relatively lower resource use in the intermediate (‘urgent’) level compared to the MTS and ESI. The median ED length of stay (LOS) for the entire cohort was 2.7 h [1.8–3.8], with the shortest LOS observed among patients triaged as most or least urgent across all triage systems.

Table 3 Resources and time in the emergency department

Hospital admission and mortality

Table 4 presents the clinical outcomes per urgency level for each triage system. A higher percentage of patients in the ‘most urgent’ level of the NTS cohort was discharged home (3.5%) compared to the MTS (0.3%) and ESI (0.1%). Additionally, there was a remarkably high percentage of in-hospital mortality in the least urgent category for the NTS (12.4%), when compared with the MTS (6.3%) and the ESI (0.8%). Furthermore, hospitalization and in-hospital mortality rates were lower in the lowest urgency level of the ESI compared to the MTS and NTS. The median hospital LOS in the total cohort was 3.0 days [1.0–7.0]. The MTS and ESI showed the longest hospital LOS in the ‘most urgent’ category, whereas the NTS had the longest hospital LOS in the intermediate urgency categories (‘urgent’ and ‘very urgent’). AORs for in-hospital mortality and hospital admission are displayed in Figs. 1 and 2 and Additional File 5. Robustness check showed that the association between urgency levels and outcomes remained consistent after adjusting for different covariates. VIF values were approximately one, indicating that multicollinearity was not a concern in the regression models. The risk (AORs) for in-hospital mortality and hospital admission increases with incrementing urgency levels in all triage systems. However, the ESI demonstrated the most pronounced increase in risk for in-hospital mortality and hospitalization with increasing urgency levels.

Table 4 Clinical outcomes of emergency department patients

Fig. 1

The association between urgency levels and in-hospital mortality. Legend: AOR: Adjusted Odds Ratio; MTS: Manchester Triage System; ESI: Emergency Severity Index; NTS: Netherlands Triage Standard. AOR adjusted for age, sex, top ten presenting complaints and hospital type (general (reference category) and academic). Reference group: not urgent. Top ten presenting complaints: (1) Extremity problems; (2) Feeling unwell; (3) Abdominal pain; (4) Dyspnea; (5) Chest pain; (6) Trauma major; (7) Wounds; (8) Urinary problems; (9) Falls; (10) Other (reference category)

Fig. 2

The association between urgency levels and hospital admission. Legend: AOR: Adjusted Odds Ratio; MTS: Manchester Triage System; ESI: Emergency Severity Index; NTS: Netherlands Triage Standard. AOR adjusted for age, sex, top ten presenting complaints and hospital type (general (reference category) and academic). Reference group: not urgent. Top ten presenting complaints: (1) Extremity problems; (2) Feeling unwell; (3) Abdominal pain; (4) Dyspnea; (5) Chest pain; (6) Trauma major; (7) Wounds; (8) Urinary problems; (9) Falls; (10) Other (reference category)

Discussion

Previous studies have shown considerable variability in the performance of different triage systems in predicting patient outcomes such as hospital admission and mortality. This study suggests that the ESI may be more effective in distinguishing between patients with low and high urgency compared to the MTS and NTS. The substantial proportion of ED patients who are hospitalized and die in the low urgency levels suggest a higher risk of undertriage in the MTS and NTS, which may impact patient outcomes and resource allocation.

Previous systematic reviews assessing the performance of the ESI and MTS, conducted by Hinson et al. (15 studies on the ESI and 14 on the MTS) and Zachariasse et al. (21 studies on the ESI and 15 on the MTS), concluded that there is no clear preference for either system, as their overall performance appears comparable [7, 8]. However, many of these observational studies were limited by single-center designs, small or specific patient populations, and differences in settings and healthcare systems, limiting their generalizability [7, 8, 13, 25, 26]. Furthermore, direct head-to-head comparisons of the MTS and ESI within a single study and across large, diverse patient populations remain scarce [25, 26]. Our findings add to the existing evidence by demonstrating that the adjusted risk of hospital admission increased significantly more with increasing urgency levels in the ESI compared to the MTS and NTS, suggesting that urgency levels of the ESI better discriminate the need for hospitalization. This aligns with previous findings by van der Wulp et al. [25] Additionally, our study extends prior research by examining in-hospital mortality and adjusting for presenting complaints, factors that were not considered in earlier studies.

The few studies suggesting that the NTS is a valid triage system are limited by small sample sizes, single-center designs, or its reliance on case scenarios [19, 20]. These studies also report an elevated risk of both under- and overtriage in the NTS, a finding consistent with our results [19, 20]. By being the first multicenter study to comprehensively assess the performance of the NTS in ED triage and directly compare it to other triage systems, our study provides a broader and more generalizable perspective on its effectiveness.

The pattern of low resource utilization, hospitalization, and in-hospital mortality in the lowest urgency level and progressively higher levels in the higher urgency levels, suggests that the ESI more effectively differentiates between low- and high-acuity patients compared to the MTS and NTS. The ESI appears to be least affected by undertriage as it shows the lowest rates of resource utilization, hospital admissions, and in-hospital mortality in the lowest urgency level compared to the MTS and NTS, which exhibit a substantial risk for hospitalization and mortality in the lowest urgency level. Lower resource utilization in the lowest urgency levels of the ESI, compared to the MTS and NTS, would be expected as the ESI incorporates resource utilization into its triage algorithm. The anticipated need for multiple resources during initial assessment generally results in patients being assigned to at least an intermediate urgency level, or to a higher level if vital signs are abnormal [9]. Additionally, a higher proportion of patients were triaged as ‘urgent’ in the ESI (68.7%) compared to the MTS (50.1%) and NTS (35.4%). This suggests that the ESI system may be more effective in identifying patients who require more intensive resources for assessment, reflecting an appropriate triage process where additional resources are needed to determine whether a patient can be safely discharged.

Furthermore, the NTS possibly has an elevated risk of overtriage, as suggested by a greater proportion of patients discharged home from the highest urgency levels. This could result in greater strain on the ED, potentially depleting resources and affecting care for other patients.

There are indications that triage systems may perform differently across various age groups [14, 15, 27]. While our study evaluated the performance of triage systems across all age groups, future research should compare how these systems perform specifically within both pediatric and geriatric populations. Furthermore, as EDs experience increasing crowding, implementing a valid triage system is crucial, and adding additional triage methods could be beneficial. A simple triage score that incorporates mobility, mental status, and oxygen saturation has been shown to identify twice as many patients at low risk of early death compared to the ESI [28]. Enhancing the ESI by integrating measures of mobility and mental status into its protocol could potentially improve its performance.

This study has several limitations. First, a retrospective observational study is susceptible to potential documentation or data entry errors. However, as the data entry process was largely automated, the risk of misregistration was minimized. Second, the ESI triage system was used in only one hospital, which may affect the generalizability of the findings to other settings. Nonetheless, this hospital includes two ED locations and a large study population, which helps mitigate this concern. In addition, there could be potential case-mix differences between different hospital populations. To mitigate this, hospital type was included as a covariate in the multivariable logistic regression model, reducing the potential confounding effect of hospital type on the association between triage category and outcome. However, we believe that the case-mixes in terms of comorbidity and complexity are comparable between the hospitals. As shown in Table 2, patient characteristics such as age, sex, arrival by ambulance (a measure of disease severity), and presenting complaints are similar across hospitals. Additionally, Table 3 demonstrates that the proportion of patients undergoing blood tests, radiological tests, or interventions relative to the total number of patients in the ESI, MTS, and NTS groups is also comparable, further suggesting that case-mix differences are unlikely to explain the observed results.

Furthermore, the synchronization of presenting complaints of the MTS, ESI and NTS to enable comparison, may have introduced some categorization errors. Yet, no substantial differences were observed between the groups after merging the presenting complaints.

A key strength of this study is its multicenter design, which includes both academic and general hospitals across multiple locations. The large cohort size in this study further strengthens the generalizability of the findings. The use of a consistent study design and similar outcome measures to compare triage systems within the same healthcare system reduces the influence of external factors, such as inter-country differences and variations in healthcare systems, on triage performance. Furthermore, conducting a multivariable regression analysis, with adjustments for potential confounders such as age, gender, presenting complaint, and hospital type, enhances the reliability of our findings.

Conclusion

The sharper increase in risk for hospital admission and mortality with increasing urgency level suggests that the ESI more effectively discriminates between low and high urgency levels. The substantial proportion of ED patients who are hospitalized and die in the low urgency levels suggest a higher risk of undertriage in the MTS and NTS. Future studies should explore performance differences between these triage systems across various age groups.