Experiment | Training set | Testing set | Classification task | MCC [CI: 95%] | BAC [CI: 95%] | ROC AUC [CI: 95%] | Sens [CI: 95%] | Spec [CI: 95%] | PPV (prevalence) [CI: 95%] | NPV (prevalence) [CI: 95%] | PPV (standard) [CI: 95%] | NPV (standard) [CI: 95%] | TN | FP | FN | TP | MCC p-value (vs. w/ GT) |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
A1 | ADNI MPRAGE (N=295) | ADNI MPRAGE (N=128) | HC vs. MCI | 0.056 [−0.159;0.259] | 52.8% [42.1%; 62.9%] | 58.2% [46.5%; 70.1%] | 50.0% [35.0%; 64.6%] | 55.6% [41.5%; 69.6%] | 33.3% [21.0%; 48.5%] | 71.5% [59.0%; 81.6%] | 53.0% [37.4%; 68.0%] | 52.7% [39.0%; 66.3%] | 25 | 20 | 22 | 22 | 0.060 |
HC vs. AD | 0.814 [0.679; 0.927] | 90.8% [84.1%; 96.2%] | 97.3% [93.6%; 99.6%] | 94.9% [87.2%; 100.0%] | 86.7% [76.2%; 95.5%] | 81.7% [69.6%; 93.3%] | 96.4% [90.5%; 100.0%] | 87.7% [78.6%; 95.7%] | 94.4% [85.6%; 100.0%] | 39 | 6 | 2 | 37 | 0.974 | |||
MCI vs. AD | 0.588 [0.424; 0.751] | 79.0% [70.7%; 87.5%] | 87.9% [79.9%; 95.3%] | 89.7% [78.6%; 97.7%] | 68.2% [54.3%; 82.4%] | 80.0% [70.9%; 88.7%] | 82.4% [64.2%; 96.2%] | 73.8% [63.2%; 84.7%] | 86.9% [71.7%; 97.3%] | 30 | 14 | 4 | 35 | 0.627 | |||
HC vs. MCI vs. AD | 0.350 [0.227; 0.480] | 57.7% [50.2%; 65.2%] | 76.6% [70.8%; 82.3%] | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 0.674 | |||
A2 | ADNI IR-SPGR (N=106) | ADNI IR-SPGR (N=46) | HC vs. MCI | 0.087 [−0.267; 0.457] | 53.8% [38.7%; 70.4%] | 69.2% [51.2%; 86.0%] | 28.6% [7.1%; 54.5%] | 78.9% [61.1%; 95.0%] | 37.5% [7.5%; 82.8%] | 71.4% [59.8%; 82.5%] | 57.5% [15.4%; 91.6%] | 52.5% [39.7%; 67.6%] | 15 | 4 | 10 | 4 | 0.860 |
HC vs. AD | 0.607 [0.313; 0.867] | 79.4% [65.0%; 92.9%] | 93.5% [82.7%; 100.0%] | 69.2% [44.4%; 92.9%] | 89.5% [75.0%; 100.0%] | 80.5% [52.6%; 100.0%] | 82.3% [68.3%; 95.7%] | 86.8% [64.0%; 100.0%] | 74.4% [57.4%; 93.4%] | 17 | 2 | 4 | 9 | 0.384 | |||
MCI vs. AD | 0.408 [−0.028; 0.742] | 70.1% [51.5%; 86.7%] | 77.5% [55.1%; 94.0%] | 61.5% [33.3%; 86.7%] | 78.6% [50.0%; 100.0%] | 80.3% [48.5%; 100.0%] | 59.1% [34.6%; 84.2%] | 74.2% [40.0%; 100.0%] | 67.1% [42.8%; 88.3%] | 11 | 3 | 5 | 8 | 0.737 | |||
HC vs. MCI vs. AD | 0.263 [0.054; 0.491] | 49.8% [37.1%; 62.9%] | 77.1% [66.4%; 87.5%] | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 0.996 | |||
A3 | ADNI MPRAGE (N=423) | ADNI IR-SPGR (N=147) | HC vs. MCI | 0.268 [0.093; 0.423] | 61.9% [54.2%; 69.4%] | 73.5% [62.9%; 82.7%] | 88.4% [77.8%; 97.4%] | 35.5% [23.9%; 48.3%] | 37.8% [31.2%; 45.5%] | 87.4% [70.8%; 97.7%] | 57.8% [50.6%; 65.3%] | 75.4% [51.8%; 94.9%] | 22 | 40 | 5 | 38 | 0.816 |
HC vs. AD | 0.745 [0.612; 0.867] | 87.6% [80.7%; 94.0%] | 96.3% [92.7%; 98.9%] | 88.1% [77.8%; 97.2%] | 87.1% [77.9%; 95.1%] | 81.0% [68.8%; 92.5%] | 92.1% [84.9%; 98.2%] | 87.2% [77.9%; 95.2%] | 88.0% [77.8%; 97.1%] | 54 | 8 | 5 | 37 | 0.367 | |||
MCI vs. AD | 0.647 [0.480; 0.805] | 82.4% [74.1%; 90.1%] | 87.4% [79.8%; 90.1%] | 83.3% [71.4%; 93.8%] | 81.4% [68.9%; 92.9%] | 86.4% [76.5%; 94.9%] | 77.5% [63.0%; 91.3%] | 81.7% [69.7%; 92.9%] | 83.0% [70.7%; 93.7%] | 35 | 8 | 7 | 35 | 0.496 | |||
HC vs. MCI vs. AD | 0.424 [0.312; 0.533] | 61.7% [54.6%; 68.8%] | 81.6% [75.6%; 87.3%] | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 0.618 | |||
A4 | ADNI IR-SPGR (N=147) | ADNI MPRAGE (N=423) | HC vs. MCI | 0.178 [0.074; 0.287] | 57.1% [53.0%; 61.7%] | 65.6% [59.5%; 72.0%] | 26.9% [20.4%; 34.1%] | 87.2% [81.9%;92.3%] | 48.2% [33.3%; 66.2%] | 72.9% [69.9%; 76.0%] | 67.8% [53.0%; 81.6%] | 54.4% [50.7%; 58.3%] | ## | 19 | ## | 39 | 0.462 |
HC vs. AD | 0.711 [0.631; 0.792] | 85.4% [81.2%; 89.4%] | 93.5% [90.7%; 95.9%] | 82.2% [75.0%; 88.1%] | 88.6% [83.3%; 93.3%] | 81.9% [73.8%;89.2%] | 88.8% [84.2%; 92.6%] | 87.8% [81.8%; 92.9%] | 83.3% [76.9%; 88.7%] | ## | 17 | 23 | ## | 0.893 | |||
MCI vs. AD | 0.454 [0.353; 0.551] | 71.7% [66.6%; 76.5%] | 81.4% [76.2%; 86.3%] | 87.6% [81.4%; 92.9%] | 68.3% [56.5%; 70.9%] | 73.7% [68.9%; 78.5%] | 76.16% [64.6%; 86.4%] | 66.5% [61.0%; 72.0%] | 81.8% [72.1%; 90.0%] | 81 | 64 | 16 | 113 | 0.114 | |||
HC vs. MCI vs. AD | 0.372 [0.307; 0.431] | 57.0% [53.4%; 60.3%] | 75.5% [72.2%; 78.8%] | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 0.492 | |||
A5 | ADNI IR-SPGR and ADNI MPRAGE (N=379) | ADNI IR-SPGR and ADNI MPRAGE (N=164) | HC vs. MCI | 0.265 [0.085; 0.441] | 62.6% [54.0%; 71.0%] | 64.4% [53.7%; 74.1%] | 47.2% [34.5%; 60.4%] | 78.0% [66.7%; 87.7%] | 48.7% [31.5%; 68.5%] | 76.9% [69.7%; 83.3%] | 68.2% [50.9%; 83.1%] | 59.6% [50.5%; 68.9%] | 46 | 13 | 28 | 25 | 0.404 |
HC vs. AD | 0.789 [0.669; 0.892] | 89.5% [83.7%; 94.7%] | 96.0% [92.3%; 98.8%] | 94.2% [87.0%; 100.0%] | 84.7% [75.4%; 93.8%] | 79.4% [68.9%; 91.0%] | 95.9% [90.3%; 100.0%] | 86.0% [78.0%; 94.2%] | 93.6% [85.3%; 100.08%] | 50 | 9 | 3 | 49 | 0.883 | |||
MCI vs. AD | 0.592 [0.433; 0.734] | 79.1% [71.0%; 86.2%] | 88.1% [80.6%; 93.6%] | 88.5% [79.2%; 96.2%] | 69.8% [55.9%; 81.1%] | 80.6% [71.8%; 87.8%] | 81.1% [65.5%; 93.8%] | 74.6% [64.2%; 83.6%] | 85.9% [72.9%; 95.5%] | 37 | 16 | 6 | 46 | 0.774 | |||
HC vs. MCI vs. AD | 0.438 [0.330;0.541] | 62.1% [55.5%; 68.2%] | 77.7% [72.2%; 82.6%] | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 0.505 |