Which AI-image detectors actually work?
A neutral, reproducible measurement of off-the-shelf synthetic-image detectors on a balanced synthetic-face corpus (1,200 real + 1,200 generated, 12 demographic cells). We report tie-correct AUC, robustness under platform re-encoding, and demographic fairness. No vendor scores, no marketing, just the same numbers our evaluation engine serves. Want your own model measured? Submit a detector.
On the clean corpus the strongest detector, Deepfake-Detect-Siglip2, reaches only AUC 0.717. The weakest, Corvi2023, sits at 0.243, below chance, i.e. systematically inverted on this distribution.
Want a specific slice? Query any detector under any condition and demographic cell.
Run a live queryLeaderboard, clean corpus
Overall tie-correct AUC with the detector's own operating point (FNR = fakes missed, FPR = real images falsely flagged) on subset 2026-05-31_perturbation_full_v1.
| # | Detector | AUC | Missed fakes | False alarms | N |
|---|---|---|---|---|---|
| 1 | Deepfake-Detect-Siglip2prithivMLmods | 0.717 | 85.9% | 3.0% | 1200/1200 |
| 2 | xceptionSCLBD | 0.711 | 48.5% | 26.3% | 1200/1200 |
| 3 | SMOGY-Ai-images-detectorSmogy | 0.700 | 0.3% | 92.1% | 1200/1200 |
| 4 | Deep-Fake-Detector-v2-ModelprithivMLmods | 0.661 | 47.9% | 31.3% | 1200/1200 |
| 5 | f3netSCLBD | 0.572 | 58.1% | 34.2% | 1200/1200 |
| 6 | efficientnetb4SCLBD | 0.537 | 51.0% | 44.9% | 1200/1200 |
| 7 | fusiongrip-unina | 0.340 | 20.9% | 92.3% | 1200/1200 |
| 8 | Corvi2023grip-unina | 0.243 | 100.0% | 0.8% | 1200/1200 |
AUC below 0.5 means the detector's score is anti-correlated with truth on this corpus (it tends to call generated faces more “real” than the reals).
Demographic fairness (clean corpus)
The AUC gap between each detector's best- and worst-performing demographic cell (skin tone × gender). A strong pooled number can hide a subgroup that falls toward random. Sorted by gap, widest first.
| Detector | Fairness gap | Worst cell | Best cell |
|---|---|---|---|
| Deep-Fake-Detector-v2-Model | 0.434 | very light/female 0.378 | brown/female 0.812 |
| Corvi2023 | 0.403 | intermediate/male 0.078 | dark/female 0.480 |
| Deepfake-Detect-Siglip2 | 0.347 | dark/male 0.521 | tan/female 0.868 |
| f3net | 0.315 | intermediate/female 0.411 | tan/female 0.726 |
| SMOGY-Ai-images-detector | 0.283 | brown/male 0.571 | intermediate/female 0.854 |
| xception | 0.259 | very light/female 0.609 | tan/female 0.868 |
| efficientnetb4 | 0.237 | light/male 0.460 | tan/female 0.697 |
| fusion | 0.196 | light/female 0.240 | brown/female 0.436 |