Skip to main content

Table 3 Direct comparison and performance evaluation of the other four algorithms on the testing and validation set

From: Utilization of non-invasive ventilation before prehospital emergency anesthesia in trauma – a cohort analysis with machine learning

 

REPTree

Bayes network

Testing

Validation

p

Testing

Validation

p

Correctness (%)

89.64 [87.93–91.34]

91.36 [90.59–92.14]

0.07

94.53 [93.61–95.44]

94.76 [94.66–94.86]

0.34

Kappa

0.76 [0.73–0.8]

0.8 [0.78–0.82]

0.1

0.88 [0.86–0.9]

0.88 [0.88–0.88]

0.67

MCC

0.77 [0.73–0.81]

0.8 [0.78–0.82]

0.11

0.88 [0.86–0.9]

0.88 [0.88–0.88]

0.7

Threshold

0.77 [0.73–0.81]

0.71 [0.62–0.81]

0.29

0.54 [0.51–0.56]

0.56 [0.55–0.57]

0.09

AUC-ROC

0.91 [0.9–0.93]

0.94 [0.93–0.94]

0.04*

0.96 [0.95–0.96]

0.96 [0.95–0.96]

0.88

Sensitivity

0.81 [0.78–0.85]

0.84 [0.83–0.86]

0.13

0.91 [0.9–0.93]

0.91[0.91–0.91]

0.4

PPV

0.87 [0.84–0.9]

0.88 [0.87–0.9]

0.48

0.92 [0.89–0.95]

0.93 [0.93–0.93]

0.49

PRC-area

0.86 [0.83–0.9]

0.88 [0.86–0.89]

0.44

0.95 [0.95–0.96]

0.96 [0.96–0.96]

0.59

Specificity

0.94 [0.92–0.96]

0.95 [0.94–0.95]

0.34

0.96 [0.95–0.98]

0.97 [0.97–0.97]

0.4

NPV

0.91 [0.89–0.92]

0.93 [0.92–0.93]

0.04*

0.96 [0.95–0.97]

0.96 [0.96–0.96]

0.74

PRC-area

0.93 [0.91–0.94]

0.96 [0.95–0.96]

< 0.01*

0.96 [0.95–0.96]

0.95 [0.95–0.95]

0.16

 

Random Forest

Multilayer perceptron

Testing

Validation

p

Testing

Validation

p

Correctness (%)

93.53 [92.79–94.28]

93.15 [92.87–93.43]

0.32

79.52 [78.06–80.98]

83.06 [81.35–84.77]

< 0.01*

Kappa

0.85 [0.84–0.87]

0.84 [0.84–0.85]

0.24

0.51 [0.48–0.55]

0.58[0.53–0.63]

0.03*

MCC

0.85 [0.84–0.87]

0.84 [0.84–0.85]

0.17

0.53 [0.5–0.57]

0.6 [0.56–0.64]

0.02*

Threshold

0.51 [0.5–0.53]

0.51 [0.5–0.52]

0.69

0.56 [0.51–0.61]

0.52 [0.5–0.54]

0.17

AUC-ROC

0.96 [0.95–0.97]

0.95 [0.95–0.95]

0.02*

0.81 [0.79–0.83]

0.83 [0.81–0.85]

0.11

Sensitivity

0.86 [0.84–0.88]

0.88 [0.87–0.88]

0.22

0.58 [0.51–0.65]

0.61 [0.52–0.7]

0.59

PPV

0.94 [0.93–0.95]

0.91 [0.9–0.92]

< 0.01*

0.75 [0.66–0.84]

0.85 [0.8–0.9]

0.06

PRC-area

0.96 [0.95–0.96]

0.94 [0.94–0.95]

0.01*

0.76 [0.74–0.78]

0.79 [0.77–0.82]

0.06

Specificity

0.97 [0.97–0.98]

0.96 [0.95–0.96]

< 0.01*

0.9 [0.86–0.95]

0.95 [0.93–0.97]

0.06

NPV

0.94 [0.92–0.95]

0.94 [0.94–0.94]

0.51

0.81 [0.78–0.84]

0.85 [0.81–0.88]

0.14

PRC-area

0.96 [0.95–0.97]

0.96 [0.95–0.96]

0.39

0.86 [0.83–0.9]

0.88 [0.86–0.89]

0.46

  1. 95% confidence interval given in parentheses, abbreviations: AUC-ROC, area under the receiver operating characteristic curve; PPV = positive predictive value; NPV = negative predictive value; MCC, Matthews correlation coefficient; PRC-area, precision-recall-area (given for the prediction and exclusion of non-invasive ventilation); REPTree = reduced error pruning tree, *statistically significant value (p < 0.05)
Back to article page

Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine

ISSN: 1757-7241