We attack your defense before the fraudsters do.
Margen is the independent red team for deepfake detection. You bring the detection system you have bought or built; we find out, in plain terms, whether it actually stops what is coming.
Three steps from question to verdict.
We pin down the threat you actually face.
We agree on the attacks that matter for your deployment, the conditions your media arrives in, and the groups you need covered. No generic test, no surprises.
We hit your detector with fresh, realistic fraud.
Your detector meets attacks it has never seen, re-compressed the way platforms re-compress every upload, across the full demographic range. The same thing an adversary would do, done first by us.
You get a clear verdict you can act on.
A plain-language report card: where the detector holds, where it fails, and the worst case by group. Where it fails, we hand back exactly what broke it so it can be fixed.
Six weeks. Five milestones. One report on the other side.
- W1
Kickoff and scope lock
Attack classes, demographic axes, and the bar for a finding are agreed and frozen.
- W2
Integration and calibration
Your detector is wired into the harness. Baseline performance captured before any perturbation.
- W3 to W4
Adversarial runs
Benchmark and perturbation pipeline executed. Every decision logged for replay.
- W5
Analysis and bypass recipes
Per-group margins of error computed. Failures annotated with the recipe that surfaced them.
- W6
Verdict and engineering debrief
Evaluation report delivered. Working session with engineering. Verdict signed.
Co-delivered engagements are matched to the host scope and may run shorter or longer.
The number that sold you the detector is not the number that protects you.
Three things quietly turn a strong benchmark score into a real-world miss. An honest evaluation has to account for all of them.
The core challenge
Why a strong-looking detector still lets fraud through.
Trained on yesterday
Detectors learn from the generators that existed when they were built. New models ship every month, and the detector has never seen them.
Tested in a lab
Vendor numbers come from clean, pristine images. Real fraud arrives compressed, resized, and re-encoded by the platforms it passes through.
Measured on the average
A strong overall score can hide groups the detector barely catches. The average looks fine while a whole subgroup is an open lane.
You receive a report. Here is what is inside it.
Verdict
Pass, conditional, or fail. Definitions set during scope lock, not after.
Method trace
Every claim traceable to a dataset version and a pipeline hash. Reproducible end to end.
Per-group breakdown
Demographic and platform groups with margin of error. The number you would actually defend.
Bypass recipes
Every failure annotated with the recipe that surfaced it. Engineering can replay.
EVAL-2026-043
6-week audit · signed by the lead researcher
Per-group true-positive rate
Bypass recipes · 3 critical
- F-1141Codec re-encode · dark, female
- F-1138Document presentation · light, male
- F-1122Resampled synthetic · intermediate, female
p. 01 / 22 · illustrative
Find your blind spot before someone else does.
Tell us what you are protecting and which detection system is protecting it. We will scope an evaluation and a timeline.