Independent red team for biometric systems

We break the deepfake detectors your fraud defense relies on.

Margen is the independent red team for your deepfake defense. We attack the detection you have bought or built with the fraud that is actually circulating, then tell you where it holds, where it fails, and what to do about it.

Request an evaluation Interactive demo

Best fit for organizations holding large amounts of biometric data.

Detector report card

vendor-x v4.2

CONDITIONAL

Unseen generatorsFails
Platform-realistic conditionsDegrades
Per-group fairnessGap found
Clean benchmarkPasses

Illustrative. Real engagements report the exact failure path.

Where the human layer ends

At scale, no one is checking by hand. The system is.

A real face and its AI impersonation of the same identity. Drag to compare.

High-volume identity flows, onboarding, verification, remote interviews, move far more traffic than any team can review by hand. And the best fakes now slip past trained reviewers anyway. The human layer cannot hold the line.

So your detection system is the fraud-prevention layer. These systems run several checks at once, and a weak spot at any one of them lets the fraud through.

That technical control is what we evaluate. We find where it fails, before an attacker does.

The core challenge

Why a strong-looking detector still lets fraud through.

01 / Recency

Trained on yesterday.

Detectors learn from the generators that existed when they were built. New models ship every month, and the detector has never seen them.

02 / Realism

Tested in a lab.

Vendor numbers come from clean, pristine images. Real fraud arrives compressed, resized, and re-encoded by the platforms it passes through.

03 / Fairness

Measured on the average.

A strong overall score can hide groups the detector barely catches. The average looks fine while a whole subgroup is an open lane.

The problem, measured

The detectors that scored perfect collapsed the hardest.

Detection score, where 1.00 is perfect and 0.50 is a coin flip.

Clean lab test

1.00score

Two open-source detectors that hit a perfect score on a clean test.

Decayto a coin flip

Real conditions

0.34score

The same two, re-tested against fresh attacks and the compression real platforms apply. Six other detectors slipped too, but far less.

Source: Margen open-source detector benchmark · 14 detectors

Read the benchmark

What we test

We test against attacks your detector has never seen.

Fraudsters do not use last year's models. We evaluate against generators held out of your training and add new ones as threats emerge, so the score reflects tomorrow's attack.

What we offer

Three ways to red-team a detector.

All three draw on the same dataset and the same methodology. They differ in whether you run it yourself or we run the engagement, and who owns the customer relationship.

Evaluation

You submit a detector. We red-team it.

Initiated by: The detection vendor
Duration: 4 to 6 weeks, fixed scope
Deliverable: Report: verdict, per-group results, and the recipes that broke it
Sensitive data: Runs inside your perimeter on your own biometric reference data
Used for: Procurement, marketing-claim validation, pre-release QA

Request an evaluation

Co-delivered

Your red team brings us in for the technical layer.

Initiated by: A red-team or security-awareness partner
Duration: Matches the host engagement
Deliverable: Joint report, human and technical layers
Used for: Enterprise security audits, joint engagements

Become a partner

Self-service

You pull the data and run it yourself.

Initiated by: You, self-serve
Duration: On demand, ongoing
Deliverable: Labeled attack data via API, by cell, generator, condition
Used for: Internal QA, CI regression, pre-release testing

Get an API key

Attack-Data API

Purpose-built attack datasets, one per threat model.

Pull any of them through the same API, filtered by demographic cell, generator, and condition. Two live today, more on the roadmap, one schema across all of them so you can compare rigor before committing.

See all data cards

SAMPLE

MG-SYN-01Available

Synthetic Face Benchmark

Full-face synthesis with identity conditioning, evaluated across demographics and platform conditions.

Passport and ID Face-Injection

Identity-preserving injection attacks framed for the passport, ID, and remote-KYC threat model.

Video face-swap attacks with per-frame swap provenance and temporal labels.

Coming soon

MG-PUP-01Coming soon

i2v Puppeteering

A still portrait driven into motion, image-to-video puppeteering.

Coming soon

MG-LIV-01Coming soon

Active Liveness PAD

Presentation-attack instruments for active liveness checks.

Who we serve

Five teams, one measurement layer.

+Somewhere else? Tell us your use case

01 / 05Vendor

“The third-party red team that helps you close the deal.”

Your buyers ask for proof that goes beyond your own benchmark. We are the independent red team that supplies it: an evaluation grounded in a corpus your team did not assemble and a method your team did not design, so the number holds up in the room where the deal is won.

What we measure for them

Per-group performance. Demographic and platform breakdowns of every score.
Bypass recipes. Every failure annotated with the recipe that surfaced it.
Pre-release QA. A second pair of eyes before you ship.

Request an evaluation Read the full brief

Why Margen

Three commitments the measurement layer cannot exist without.

01 / Coverage

Coverage that tracks the threat.

Our evaluation corpus expands toward the frontier generators adversaries are adopting, so the benchmark keeps pace with the attack.

02 / Method

Reproducible methodology.

Every claim is backed by a dataset, a documented pipeline that recompresses media the way platforms do, and a pre-registered statistical methodology. Results can be independently re-run by anyone with corpus access on request.

03 / Fit

Context-fit evaluation.

We tailor the assessment to the threat the enterprise actually faces. The methodology is rigorous within each context, not generic across them.

Versus the alternatives

Not an internal team. Not a generalist pentest.

A detector is a specialized control, and evaluating it takes a specialized, independent adversary. Here is how an engagement compares to the alternatives most teams reach for first.

Capability	Margen	Internal red team	Generalist pentest
Independent of the vendor under test
Deepfake-specific attack corpus
Per-group fairness breakdown
Platform-realistic conditions
Pre-registered, reproducible method
Hands back the breaking recipe

YesPartialNo

14: detectors evaluated
12: demographic groups covered per detector
1.00 to 0.34: top detectors, from perfect to below a coin flip
0: detectors we sell, by design

What our research shows

Near-perfect on paper, near-random under attack.

Read the research

Finding

99% to a coin flip

Detectors that scored near-perfect on a clean benchmark dropped to barely better than a coin flip once we sent attacks they had not seen, re-compressed the way a real platform would. The pitch number was not the production number.

Read the research Finding

Hidden blind spots

Strong overall scores concealed groups the same detector barely caught. For a buyer that is both an open fraud lane and a fairness liability, invisible until someone stratifies the results.

Read the research Finding

Stale by the quarter

Detectors that lead today degrade as new generators ship. A validation from six months ago does not tell you whether you are covered against the attacks circulating now.

Read the research

Find your blind spot before someone else does.

Submit a model for evaluation, or add the detection layer to a red-team engagement. We return a per-group report card showing both kinds of mistake and the margin of error on each, and where a detector fails, the recipe that broke it. For buyers, we can point you to the detection that actually holds for your use case.

Request an evaluation How it works