TMGoat is a benchmark and dojo for threat-modeling tools and engineers — 30 realistic systems across 10 sectors, each seeded with deliberate design flaws, a hidden answer key, and a scoring harness that grades you on recall (did you find the real threats?) and precision (did you avoid the noise?).
// like WebGoat & TerraGoat — intentionally vulnerable. teaching material, not production.
TMGoat makes the expected output first-class. Every fixture ships an expert-authored reference model, so a tool — or an engineer — can be measured, not just demoed. Run it over the inputs, export findings, get a number back.
Publishing every solution next to its question lets tools memorize the test. So the corpus is split.
The easy and moderate tiers. Model from the inputs, use graduated hints if you're stuck, then self-score against the published answer key.
The difficult tier. Inputs are public; answer keys live in a private vault and are scored by a fixed harness — so no tool is graded on answers it has seen.
A six-component app can hide a nastier flaw than a thirty-component one. Every fixture is tagged on three independent axes — so the benchmark measures reasoning, not parsing.
How much system there is to reconstruct from the inputs.
Whether the risk is textbook or a design-level trap.
From rich and corroborating to sparse — or deliberately lying.
A tool that just runs STRIDE on each box misses the race condition, the fail-open control, the over-trusted channel, the doc that lies.
Those design threats are where the points — and the risk — live.
Open a challenge, model it from inputs/, self-score, then study the reference. Level up per fixture.
Run your tool over the inputs, export findings, and score recall / precision against the key.
# practice tier — keys ship in-repo python harness/score.py \ --expected fixtures/<sector>/<tier>/solution/threat-model.yaml \ --findings your-findings.json
Recall and precision on the 10 held-out fixtures, scored by the shared harness. Submissions open with the benchmark flow — open an issue on GitHub for early access.
| # | Tool | Recall | Precision | Chained |
|---|---|---|---|---|
Be the first entry — submit a run ↗ | ||||