Skip to main content

Table 1 PFI, BIC and SHAP success in identification of feature ranks in datasets with two-way and three-way epistatic interactions. It is expressed as the percentage of a match of a metric rank’s estimate with the true feature rank that was retrieved with the HIBACHI sensitivity analysis

From: A comparison of methods for interpreting random forest models of genetic association in the presence of non-additive interactions

Sample size 1000
Two-Way IG Three-Way IG
% of cases: 25% 50% 25% 50%
Metrics: PFI BIC SHAP PFI BIC SHAP PFI BIC SHAP PFI BIC SHAP
 F1 70% 41% 71% 91% 42% 82% 80% 38% 57% 79% 18% 68%
 F2 63% 41% 62% 90% 42% 82% 69% 33% 45% 60% 36% 52%
 F3 93% 84% 89% 78% 82% 81% 79% 33% 53% 71% 18% 73%
 F4 17% 17% 15% 15% 17% 16% 74% 55% 58% 11% 14% 11%
 F5 5% 4% 5% 4% 5% 3% 33% 31% 29% 5% 6% 4%
Sample size 10,000
Two-Way IG Three-Way IG
% of cases: 25% 50% 25% 50%
Metrics: PFI BIC SHAP PFI BIC SHAP PFI BIC SHAP PFI BIC SHAP
 F1 79% 36% 73% 83% 39% 75% 89% 42% 68% 86% 19% 76%
 F2 79% 32% 73% 83% 39% 75% 80% 34% 49% 75% 34% 56%
 F3 100% 87% 99% 81% 79% 81% 89% 33% 66% 85% 22% 70%
 F4 13% 12% 13% 44% 38% 40% 84% 62% 69% 53% 51% 52%
 F5 5% 4% 5% 15% 13% 14% 40% 38% 37% 11% 9% 10%
  1. F1, F2, etc. – feature ranks, PFI permutation feature importance, BIC build-in coefficients, SHAP shapley additive explanations, IG information gain
\