The revised Canadian Bleeding (CAN-BLEED) score for risk stratification of bleeding trauma patients: a mixed retrospective—prospective cohort study

Abstract

Background

Traumatic hemorrhage is a significant cause of morbidity and mortality. There is considerable interest in risk stratification tools to aid with early activation of intervention pathways for bleeding patients. In this study, we refine the Canadian Bleeding (CAN-BLEED) score for the prediction of major interventions in bleeding trauma patients.

Methods

We conducted a mixed retrospective-prospective cohort study. We included a retrospective cohort from the CAN-BLEED derivation study, from September 2014 to September 2017. We also conducted a prospective cohort from May 2019 to August 2021 and included both datasets for refinement of the CAN-BLEED score. The primary outcome was major intervention, defined by a composite of massive transfusion, embolization, or surgery for hemostasis. Predictors were pre-specified based on previous validation work. We used a stepdown procedure and regression coefficients to create a clinical risk stratification score. We used bootstrap internal validation to assess optimism-corrected performance.

Results

We included 1368 patients in the overall cohort. Incidence of penetrating injury was 23% and median injury severity score was 17. The overall incidence of the need for major intervention was 17%. The revised score included 8 variables: systolic blood pressure, heart rate, lactate, penetrating mechanism, pelvic instability, Focused Abdominal Sonography for Trauma positive for free fluid, computed tomography positive for free fluid, or contrast extravasation. The C-statistic for the simplified score is 0.89. A score cut-off of less than 2 points yielded a 97% (94–98%) sensitivity in ruling out the need for major intervention.

Conclusion

The revised CAN-BLEED score offers a clinically intuitive and internally validated tool with excellent performance in identifying patients requiring major intervention for traumatic bleeding. Further efforts are required to evaluate its performance with an external validation.

Background

Traumatic hemorrhage is a significant cause of morbidity and mortality among patients worldwide, with estimates suggesting that it accounts for up to 40% of all trauma deaths [1]. Uncontrolled bleeding is the most common cause of preventable mortality following military and civilian trauma [2]. Management of hemorrhage requires prompt recognition, damage control resuscitation, and early hemostasis [3]. However, severe blood loss often occurs insidiously, leading to clinical uncertainty, delayed intervention, and worse patient outcomes [4, 5].

For these reasons, there has been significant research interest in clinical decision aids to prognosticate outcomes among bleeding patients [6,7,8], especially to guide activation of massive transfusion protocols [9, 10]. However, there are concerns related to competing risk and survivorship bias for scores designed to predict massive transfusion alone [11]. The CAN-BLEED score was derived and internally validated for the prediction of any major intervention including massive transfusion protocol delivery, hemostatic surgery, or embolization in trauma patients [12]. Due to limitations in available sample size during the initial derivation study, the prespecified model was not able to incorporate all potential variables of interest.

The objective of this study is to refine the CAN-BLEED score for the prediction of major interventions in bleeding trauma patients, combining a previous retrospective cohort with a prospective cohort to attain sufficient sample size to evaluate all variables of interest.

Methods

Design and setting

We conducted a mixed retrospective-prospective observational cohort study at The Ottawa Hospital Civic Campus. The Ottawa Hospital is the designated Level 1 trauma center, university-affiliated hospital, and regional referral center for the Champlain Local Health Integration Network in Ontario, Canada. We included a retrospective observational cohort from the initial CAN-BLEED derivation study, from September 2014 to September 2017 [12]. In addition, we conducted a prospective observational cohort from May 2019 to August 2021, and included both datasets for refinement of the CAN-BLEED prediction model.

Retrospective Cohort

We included a previously published retrospective observational cohort from the CAN-BLEED derivation study, conducted from September 2014 to September 2017 [12]. In that study, we included 890 adult patients arriving alive to the Ottawa Hospital with blunt or penetrating trauma to the thorax, abdomen, or pelvis requiring trauma team activation. Patients were eligible if received from scene or from a transferring hospital within 3 h from time of injury. Patients were excluded if they had an out-of-hospital cardiac arrest or suffering from non-hemorrhagic mechanisms of injury (e.g., burn injury, drowning, or electrocution). We completed preliminary derivation and internal validation of the CAN-BLEED score to identify patients with clinically significant bleeding—defined as the need for massive transfusion, angioembolization or surgery for hemostasis. However, we did not have a sufficient number of events to incorporate all potential variables of interest such as mechanism and heart rate. Importantly, models constructed with an event per variable ratio of 10 or less are at higher risk of potential overfitting, which results in models that may deliver a strong performance during initial derivation but typically perform poorly at external validation [13]. As such, we additionally conducted a prospective observational cohort with the intention to combine datasets for refinement of the CAN-BLEED score.

Prospective cohort

In the prospective cohort, we utilized the same eligibility criteria from the prior study. We enrolled 478 adult patients arriving alive to the Ottawa Hospital with blunt or penetrating trauma to the thorax, abdomen, or pelvis requiring trauma team activation and concerns for potential traumatic bleeding between May 2019 and August 2021.

Following confirmation of patient eligibility, our study team reviewed the electronic medical record to collect standardized information on key time points, clinical characteristics, and outcomes. This was completed by a trained abstractor and an audit of the first 100 cases followed by 10% of all cases thereafter was conducted for quality assurance. Formal inter-rater reliability was not calculated. We additionally cross-referenced our abstracted data with that from our institutional trauma registry to ensure consistency with regards to case identification and calculation of the injury severity score. This is maintained by a dedicated data analyst, with a standardized input form and routine quality assurance audits consistent with all trauma centres across the province of Ontario.

The primary outcome of interest was need for major intervention, which was defined as any of the following: massive transfusion, angioembolization or surgery for hemostasis (collectively intended to represent clinically significant bleeding). To classify surgical outcomes, we reviewed all operative reports for eligible patients and included those requiring thoracotomy, laparotomy, pelvic packing or fixation, and vascular surgery for the specific intention of hemostasis. Patients receiving an exploratory operation without a therapeutic hemostatic procedure within the operative report were classified as non-events. In the initial derivation study, we utilized the classical definition for massive transfusion (≥ 10 PRBCs in 24 h). However, in this refinement study, massive transfusion protocol delivery was defined as activation of the massive transfusion protocol and receipt of at least 3 units packed red blood cells (PRBC) within the first hour (critical administration threshold) for both the retrospective and prospective datasets. This definition was utilized to minimize the survivorship or competing risk bias associated with traditional massive transfusion definitions of at least 6 or 10 units over 24 h [11]. Patients receiving angiography without embolization were also classified as not meeting the primary outcome.

Data analysis and model creation

The data analysis protocol was developed in accordance with guidelines for clinical prediction modeling [14,15,16,17]. The retrospective and prospective cohorts were merged into a single dataset and reviewed for data integrity. Similar to the methodology employed for the CAN-BLEED derivation study, we noted that the variables for Lactate (56% missing), FAST (30% missing) and CT (11% missing) were not missing at random [12]. All other variables had less than 5% missing values. Importantly, missing values for Lactate, FAST and CT were more likely to be available in patients that were clinically unwell (with abnormal findings) and more likely to be missing in those that were clinically well (with unremarkable findings). As such, when these values were missing, we considered them to be unremarkable (Lactate < 5, negative FAST, no free fluid or extravasation on CT). In accordance with best practice methodological guidance [18], the appropriateness of this assumption was investigated by comparison with imputed values generated under a multivariable imputation model. All remaining continuous variables were modelled using restricted cubic splines. We used a multiple imputation technique to impute missing values on predictor variables using the aregImpute function in the Hmisc library [19]. This procedure simultaneously imputes missing variables while determining optimal transformations among all imputation variables. The imputation model consisted of the full list of predictor variables, outcomes, and available ancillary variables (such as age, sex, and time to hospital).

We prioritized full pre-specification of the predictors, use of flexible functions for continuous predictors, and multiple imputation. Model pre-specification, based on clinical knowledge and evidence from the literature, minimizes the risk of over-fitting in small datasets and improves performance at external validation [20]. The variables reviewed for determination of model pre-specification were based on clinical importance as determined by a prior systematic review [6, 7], as well as national [8] and international surveys [21] conducted by our group. We used multiple imputation for missing values using the aregImpute function in the Hmisc library [19], which models continuous variables using restricted cubic splines with three default knots, while categorical variables are expanded as dummy variables representing all categories within a categorical variable. We used Variance Inflation Factors to examine multi-collinearity.

An initial, fully pre-specified logistic regression model was fitted on the first imputed dataset using spline functions with five knots for the two continuous variables. Next, a plot of partial associations referred to as an ANOVA plot, corrected for the number of degrees of freedom, was generated to visualize strong and weak partial associations. The strengths of association are used to inform a decision of how many degrees of freedom to allocate to each variable in the final model: strong associations are modeled with greater complexity than weak associations. The fully pre-specified model with nine predictors was fitted to the 10 multiple imputation datasets and the results were combined across the datasets using Rubin’s rules. Harrell's fast stepdown procedure was run on the previous full model [19]. Stepdown models improve practicality with fewer predictors for use in clinical practice. No variable was deleted. Analyses were conducted using R version 4.2 and SAS version 9.4 software.

Score simplification

From the stepdown model, a nomogram was generated, and the number of points associated with each predictor were obtained from the estimated regression coefficients to create a simplified score. For continuous predictors, the points were determined to correspond to midpoints of clinically meaningful intervals of values. We used a receiver operative curve analysis to assess sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and area under the curve for the simplified score as compared to the full complex model.

Internal validation

Harrell et al. presented an algorithm to assess the degree of overfitting resulting from the model building process [19]. In this algorithm, the model is estimated separately using each bootstrap sample and subsequently evaluated in the original sample to calculate optimism. Bootstrap validation maximizes sample size while minimizing potential overfitting—as opposed to split or cross validation, which is considered less statistically efficient [20]. We generated 1000 bootstrap samples and calculated optimism-corrected performance. The C-statistic for the simplified model was compared to that of the full model and the reduced model to ensure minimal change in performance.

Human ethics and consent to participate declaration

This study was approved by the Ottawa Health Science Network Review Ethics Board. A waiver of consent from participants was obtained.

Funding declaration

This study was funded by an operational grant from the Ottawa Hospital Academic Medical Organization Innovation Fund.

Results

We included 1368 patients in the overall cohort—890 patients from the retrospective derivation cohort and an additional 478 patients from the prospective cohort. The population characteristics are presented in Table 1. Most patients were males (76%) with a median age of 42 years (interquartile range [IQR] 27–58). There was a 23% incidence of penetrating injury and a median injury severity score was 17 (IQR 10–26). The overall incidence of major intervention, defined as a composite of surgery for hemostasis, embolization, or massive transfusion, was 17%. The population characteristics were similar between the retrospective derivation cohort and the prospective cohort, though we note there was a higher incidence of penetrating injury (25% vs 22%), visualized bleeding (12% vs 5%), and need for major intervention (21% vs 15%) in the prospective cohort.

Table 1 Population characteristics

Full Pre-specified and reduced models

The full pre-specified model is presented in the Supplement. The C-statistic for the full model was 0.90 and remained at 0.90 after the stepdown procedure, demonstrating excellent performance. After 1000 bootstrap samples for internal validation, the optimism-corrected C-statistic was 0.90. The reduced multivariable model, after using the stepdown procedure, included eight variables: initial systolic blood pressure, heart rate, lactate, penetrating mechanism, pelvic instability requiring fixation (as identified on clinical exam), Focused Abdominal Sonography for Trauma (FAST) positive for free fluid, and computed tomography (CT) positive for abdominopelvic free fluid or contrast extravasation. The modelled association for blood pressure and heart rate using restricted cubic splines is presented graphically in the Supplement.

Simplified score

The resulting Canadian Bleeding (CAN-BLEED) score is revised and presented in Table 2. The C-statistic for the simplified score is 0.89 confirming excellent performance.

Table 2 CAN-BLEED score composition

Measures of performance

The calibration plot (Supplement) showed excellent agreement between observed and predicted probabilities across the entire spectrum of risk. The classification performance for the revised CAN-BLEED score is presented in Table 3. A score cut-off of less than 2 points yielded a 97% (94% to 98%) sensitivity in ruling out the need for major intervention.

Table 3 Classification table

Discussion

The revised CAN-BLEED score includes the same variables as the initial model (systolic blood pressure, lactate, FAST, CT imaging, and visualized bleeding or pelvic instability), but additionally incorporates mechanism of injury and heart rate. These variables were chosen a priori based on clinical importance as determined by a systematic review [6], as well as national [8] and international surveys of trauma care providers [21]. The score shares many of the same clinical elements as the previously developed assessment of blood consumption (ABC) [9] and Trauma Associated Severe Hemorrhage (TASH) [10] scores for prediction of massive transfusion, but offers the additional advantage of being derived and internally validated specifically for a pragmatic composite outcome representative of clinically significant bleeding. Importantly, the sole utilization of massive transfusion as an outcome of interest is prone to competing risk and survivorship bias [11, 22,23,24]. A recent study of pre-hospital trauma patients demonstrated that a novel algorithm, termed the Bleeding Risk Index, using continuously recorded noninvasive vital signs may offer improved predictive performance over traditional bleeding risk scores [25]. Similar to our study, the authors utilized the critical administration threshold (3 PRBCs in 1 h) to minimize the impact of survivorship bias. However, the impact of competing risk bias and performance at external validation remains unknown. Further studies comparing the predictive performance of the refined CAN-BLEED score to existing bleeding scores is required.

In an international survey of trauma care providers from Canada, United States, Germany, Australia, and New Zealand, clinicians noted challenges in executing the risk stratification, identification of need, and delivery of meaningful interventions for bleeding trauma patients [21]. Respondents linked these challenges to systems factors such as variability in the process or efficiency of pathway activations, as well as cognitive factors such as the inherent limitations in clinical gestalt for risk stratification. Clinicians have stated a willingness to adopt a well-validated and sensitive risk stratification tool for bleeding patients but to date have expressed variable confidence in the existing decision support frameworks [21].

The revised CAN-BLEED score offers a pragmatic risk stratification tool based on variables easily available to the treating provider early in the initial assessment. The score is intentionally designed to mimic the clinician’s intuitive decision-making process, as a priori models based on clinical expertise offer improved acceptance by healthcare providers [26] and superior performance at external validation [17, 20, 27]. During the early phases of initial resuscitation, high-risk patients can be identified based on hemodynamics, clinical exam and FAST alone. If such variables are within normal limits and the clinician has a low pre-test probability of clinically important hemorrhagic injury, then further investigation is typically avoided. This decision-making mirrors the real-world assumptions utilized during derivation and internal validation of our model—whereby cases without CT imaging (30%) were specifically re-coded as unremarkable. If there is a high pre-test probability of occult injury following initial assessment, clinicians often pursue additional investigation including CT imaging—a diagnostic modality increasingly advocated for by the American College of Surgeons Committee on Trauma Guidelines on Imaging and incorporated by trauma centres into the initial evaluation of major trauma patients [28].

An objective and evidence-based approach to risk stratification has meaningful clinical, resource, and academic applications. This score was derived and validated at a Canadian tertiary care centre and reflects the practice patterns of experienced, high volume trauma care providers. While its intention is not to supplant clinical expert decision-making, its pragmatic derivation and validation based clinical and research expertise may serve to provide a reliable, evidence-based framework to homogenize decision-making for less experienced providers. In addition, it may aid clinical decision-making with regards to inter-hospital transfer assessments and activation of key intervention pathways—an important area of need identified by trauma care providers [21, 25]. In addition, it may provide a more consistent and objective assessment of risk to better define the population of “at-risk bleeding trauma patients” for enrolment into clinical trials—an endeavor that has previously been limited by imprecise event estimates resulting in inefficient sample size estimates and eventual clinical trial futility [29].

Strengths and limitations

Strengths of this study include an exhaustive evaluation of clinically important variables based on previous systematic review [6] and survey studies [8, 21], as well as adherence to best practice guidelines for derivation and validation of prediction models [14,15,16,17, 20] in order to maximize performance at external validation. In addition, we incorporated two separate datasets, including a two-year prospective observational cohort, optimizing sample size for refinement and evaluation of important variables. This study also has limitations. While the optimism-corrected performance at internal validation is impressive, this remains a single centre study at a Canadian tertiary care trauma centre. Internal validation with bootstrapping does not necessarily imply strong performance at external validation as it does not account for information or selection bias that may be not evident across different datasets [30]. In addition, bootstrap validation may deliver overly optimistic performances in small datasets due to persistent overfitting. In addition, due to meaningful missingness of data, we employed and verified assumptions of normality in such circumstances to maximize real-world pragmatism. However, the appropriateness of such assumptions can only be confirmed at external validation. We were also unable to account for time-dependent trends across the entire study period such as changes in clinical practice or composition and experience of trauma care providers. Further efforts are required to externally validate score performance at other centres in order to compare it to existing risk stratification tools and assess its potential to support clinical decision-making.

Conclusion

We conducted a mixed retrospective-prospective observational cohort study in order to refine a pragmatic bleeding risk stratification tool for bleeding trauma patients. The revised CAN-BLEED score offers a clinically intuitive and internally validated tool with excellent performance in identifying patients requiring major intervention for traumatic bleeding. Further efforts are required to evaluate its performance by external validation.