A detector test set built to fail informatively, not to score well on average.
Synthetic Face v1 pairs real portraits with ones generated by identity-conditioned diffusion (SDXL with InstantID), across twelve demographic cells and a set of platform conditions, and it labels every item with the attack behind it. The rest of this page is the case for those choices, and how to pull the data through the API.
Six deliberate choices that make it a measurement instrument, not a pile of faces.
Read top to bottom. Each row is one deliberate choice, the reason for it, and how it surfaces in the API you actually call.
- 01Pairing
Real and synthetic side by side, not two disconnected piles.
A detector's job is a decision between two hypotheses. If real portraits come from one distribution and synthetic ones from another that differs in lighting, framing, or resolution, you measure detection and distribution shift at once, then blame the model. Pairing keeps everything except authenticity as close as possible, so the score you get is the signal you want.
kind=real · kind=fake · source_real_id links the pair
REAL
SYNTHETICsame identity · one authentic, one generated - 02Stratification
Twelve demographic cells, reported as twelve results.
A single AUC across the whole set can hide a catastrophic drop on one group. Six skin-tone bands crossed with two genders give twelve cells. Skin tone is reconciled against a standard colorimetric measure so bands stay comparable across items.
filter by skin_tone × gender, read .total per cellvery_lightdarkfemalemale - 03TTP annotation
Every failure is debuggable, because every item names its attack.
Counts of correct and incorrect decisions do not tell you what to fix. Each synthetic item carries a Tactic, Technique, and Procedure tag: what the adversary tried, how they did it, and which pipeline produced it. When your detector misses, you can group the misses by TTP and target the model change that closes the gap.
TTactic, the goal (impersonation)TTechnique, the method (identity-conditioned synthesis)PProcedure, the specific pipelineGroup your misses by any row - 04Conditions
Robustness under the social-media pipelines that break detectors.
A clean crop is a laboratory condition. In production a face is JPEG re-encoded, downscaled, and pushed through the re-encode pipeline of whatever social platform it lands on. The robustness here focuses on exactly that: the Layer 2 conditions model the Facebook, Instagram, TikTok, and X pipelines, so you can point at the step where the detector fell over.
condition = clean · clean_native · layer1 · layer2cleanencoding-normalized baselinenativenative encodinglayer1JPEG · blur · noise · resizelayer2FB · IG · TikTok · X pipelinesre-croppost-platform re-framing - 05Evaluation set
Engineered to measure your detector, precisely.
Small, balanced, paired, and stratified by design. Every choice on this page compounds into one thing: a precise, reproducible read on how your detector performs, per group and per condition, that you can publish and defend. The methodology and results are written up in full.
small and balanced, sized to measure - 06Provenance
Real faces licensed for commercial use. Synthetic generated in house.
Every real portrait is licensed from commercial stock-media providers, cleared for building and evaluating detection systems, and delivered under the Margen data license. The synthetic side is generated in house, so the generator behind every item is disclosed.
source_real_id · generator · ttp per itemLicense summary- Commercial evaluation and product use
- Redistribution restricted
- No re-identification
- Provenance metadata preserved
SDXL with InstantID, targeted to fill each demographic cell.
Diffusion-based synthesis with identity conditioning. We prompt into the cell we need rather than sampling and hoping, which is why the balance holds. Buyers ask about the generator first, so we lead with it.
A single record, with the fields the API returns.
{
"benchmark": "synthetic-face-v1",
"kind": "fake",
"cell_skin_tone": "intermediate",
"cell_gender": "female",
"generator_model": "sdxl_instantid",
"condition": "layer2",
"source_real_id": "b_00184",
"ttp_tactic": "impersonation",
"ttp_technique": "identity_conditioned_synthesis"
}Query the corpus the way you would query a database.
Every dimension is a filter. Compose them, get a live .total, then pull only what you need.
GET /v1/items ?benchmark=synthetic-face-v1 &kind=fake &skin_tone=intermediate &gender=female &condition=layer2
What this dataset does not do.
If any of these matter for your threat model, tell us and we will point you to the right benchmark or start scoping one.
- 01Portrait framing only.No full-body composites and no video. Video face swap is a separate benchmark, Face-Swap Video.
- 02Perturbations on a 2,400-item subset.The layer 1 and layer 2 conditions are computed on a balanced robustness subset, not the full corpus. Deliberate; not full coverage.
- 03One generator family in v1.v1 covers a single identity-conditioned generator; it is not a census of the attack surface. More are on the roadmap.
- 04Perceptual demographic labels.Skin-tone and gender bins are perceptual labels, not biometric ground truth. Report per cell, weight accordingly.
Want this run against your own corpus?
We bring the same attack tooling into your environment and run it against your private corpus and your detector. Your data never leaves your environment, so you can test on the faces you cannot share.