| |
ECE
|
MCE
|
BS
|
P_value
|
|---|
|
NB
|
15.711 (13.557–17.914)
|
34.350 (29.275–39.800)
|
0.212 (0.199–0.228)
|
< 0.001(< 0.001- < 0.001)
|
|
NB-Platt
|
9.008 (7.919–10.647)
|
21.550 (17.475–25.800)
|
0.189 (0.181–0.197)
|
0.179 (0.055–0.389)
|
|
NB-IsoReg
|
9.820 (7.740–12.190)
|
40.000 (23.475–57.100)
|
0.208 (0.195–0.227)
|
< 0.001(< 0.001–0.057)
|
|
NB-RPR
|
8.743 (7.397–10.307)
|
21.600 (17.575–25.700)
|
0.189 (0.182–0.197)
|
0.191 (0.051–0.431)
|
|
LR
|
8.517 (7.244–10.093)
|
20.100 (16.675–25.025)
|
0.188 (0.180–0.196)
|
0.152 (0.022–0.403)
|
|
LR-Platt
|
8.981 (7.478–10.485)
|
20.900 (17.300–25.325)
|
0.189 (0.182–0.196)
|
0.215 (0.065–0.437)
|
|
LR-IsoReg
|
9.140 (6.970–11.810)
|
31.550 (20.000–50.175)
|
0.204 (0.193–0.220)
|
0.008(< 0.001–0.348)
|
|
LR-RPR
|
8.744 (7.308–10.143)
|
20.300 (16.700–24.425)
|
0.187 (0.181–0.194)
|
0.255 (0.092–0.507)
|
|
RF
|
12.740 (10.910–14.336)
|
27.200 (23.375–31.925)
|
0.201 (0.190–0.211)
|
< 0.001(< 0.001- < 0.001)
|
|
RF-Platt
|
8.998 (7.518–10.447)
|
21.100 (17.500–26.700)
|
0.192 (0.184–0.200)
|
0.156 (0.030–0.435)
|
|
RF-IsoReg
|
9.292 (7.332–11.353)
|
27.850 (20.000–40.000)
|
0.201 (0.191–0.215)
|
< 0.001(< 0.001–0.131)
|
|
RF-RPR
|
8.949 (7.387–10.524)
|
20.900 (17.400–26.025)
|
0.189 (0.182–0.196)
|
0.194 (0.061–0.458)
|
|
SVM
|
9.872 (8.317–11.777)
|
23.800 (19.000–28.925)
|
0.194 (0.185–0.204)
|
0.016(< 0.001–0.117)
|
|
SVM-Platt
|
9.077 (7.702–10.895)
|
21.750 (17.600–27.300)
|
0.192 (0.184–0.201)
|
0.169 (0.029–0.412)
|
|
SVM-IsoReg
|
9.501 (7.332–12.453)
|
30.350 (20.000–42.200)
|
0.205 (0.194–0.221)
|
0.003(< 0.001–0.249)
|
|
SVM-RPR
|
8.796 (7.362–10.439)
|
21.000 (16.775–26.550)
|
0.190 (0.183–0.199)
|
0.211 (0.064–0.471)
|
|
FFNN
|
8.238 (6.805–9.611)
|
20.150 (16.600–24.500)
|
0.184 (0.177–0.192)
|
0.244 (0.075–0.518)
|
|
FFNN-Platt
|
8.991 (7.721–10.642)
|
20.950 (16.875–26.100)
|
0.186 (0.179–0.194)
|
0.192 (0.056–0.425)
|
|
FFNN-IsoReg
|
10.866 (8.603–13.347)
|
40.550 (27.800–57.025)
|
0.211 (0.196–0.230)
|
< 0.001(< 0.001–0.003)
|
|
FFNN-RPR
|
8.703 (7.393–10.361)
|
21.400 (17.700–26.025)
|
0.185 (0.178–0.193)
|
0.227 (0.073–0.473)
|
- NB naïve Bayes, LR logistic regression, RF random forest, SVM support vector machine, FFNN feedforward neural network, Platt Platt scaling, IsoReg isotonic regression, RPR shape-restricted polynomial regression. “-Platt”, “-IsoReg” and “-RPR” represent performing probability calibration by using corresponding method. In each cell M(P25 - P75): M is the median, P25 is the 25th percentile and P75 is the 75th percentile of 500 evaluations. For each algorithm, the best performance in each column is in bold
