Skip to main content
Fig. 3 | BioData Mining

Fig. 3

From: The Matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary classification

Fig. 3

Example of ROC curve with area under the curve. a plot: This illustrative example contains a ROC plot having AUC \(=\) 0.834 that indicates a good performance in the [0; 1] interval where 0 indicates completely wrong classification and 1 indicates perfect prediction. True positive rate, sensitivity, recall, \(TPR = TP / (TP + FN)\). False positive rate, \(FPR = FP / (FP + TN) = 1 - specificity\). The AUC consists of both the green part and the red part of this plot. As highlighted by Lobo and colleagues [33], the calculation of the AUC is done by considering portions of the ROC space where the binary classifications are very poor: in the ROC space highlighted by the red square, the sensitivity and sensitivity results are insufficient (TPR < 0.5 and FPR \(\ge\) 0.5). However, this red square of bad predictions, whose area is \(0.5^2 = 0.25\), contributes to the final AUC like any other green portion of the area, where sensitivity and/or sensitivity result being sufficient instead. This red square represents 30% of the AUC \(=\) 0.834 and 25% of the whole maximum possible AUC \(=\) 1.000. How is it possible that this red portion of poor classifications contribute to the final AUC like any other green part, where at least one of the two axis rates generated good results? We believe this inclusion is one of the pitfalls of ROC AUC as a metric, as indicated by Lobo and colleagues [33] and one of the reasons why the usage of ROC AUC should be questioned. b plot: The same ROC curve with the half semicircle having \(AUC = \pi /4 \simeq 0.785\). Each point of the blue curve has \(radius = 1\) and centre in \((TPR,TNR) = (0,0)\). Point p: point with \((TPR,TNR) = (\sqrt{\frac{1}{2}},\sqrt{\frac{1}{2}}) \simeq (0.71,0.71)\)

Back to article page