Architecture & Evaluation
How Mergen intercepts every tool call in under 1ms, and the deterministic test harness that validates every policy claim.
Who governs what,
before a single call runs.
This is the authority chain: who writes a policy, and how it reaches a developer's machine. The interception diagram below tells a different, equally true story — how one call actually gets evaluated once that policy is already in place.
Platform Team
Writes policy — what every agent is and isn’t allowed to do.
My Organization
Publishes it from mergen.app. Self-hosted runtimes pick it up instantly over a live sync connection; cloud-connected runtimes pull it with mergen-server policy pull --cloud.
Mergen Runtime
Enforces it on every machine, in under a millisecond — the mechanism, not the product.
Developer
Works inside the policy — sees ALLOW, HOLD, or BLOCK on every action, in the tools they already use.
Four steps from tool call
to target runtime.
Define local security policies
Set up rules matching command patterns, file paths, and environment settings. Rules can block outright or require validation.
Intercept agent tool calls
MCP tool calls, terminal commands, and filesystem modifications pass through the local gate before execution.
Enforce deterministic rules
Mergen checks commands against safety rules and SQLite history in <1ms without LLM latency, blocking destructive actions.
HITL approval & audit logs
Pauses execution for unverified actions to request human approval. Logs every block to ~/.mergen/agent-blunders.json.
MCP Tool Ingestion
Synchronous stdio stream
IDE Execution Wrapper
Intercepts terminal runs
Git Hooks & Actions
Autonomous PR evaluation
SDK / API Integrations
Zero-trust runtime gate
Mergen
Execution & Security GatewayOS & File System
Enforced sandboxed pathsCloud Infrastructure
AWS, GCP, Terraform APIsDatabases & VPCs
Isolated query executionSlack HITL
Operator approval webhook534-Case Regression Harness
We do not claim 100% security. Mergen is evaluated against a labeled adversarial dataset (309 attacks, 225 benign) run through the exported gate decision function the live interceptor itself calls — not a test-only mirror. Our results disclose exact failure categories, open evasion gaps, and known false positives so you can calibrate policy thresholds according to your risk tolerance.
Breakdown covers the 309 attack fixtures. The 225 benign fixtures are scored separately for false positives (0 flagged) under a strict definition: any verdict other than PASS on a benign command — including a conservative HOLD — counts as a false positive. Numbers are imported from the measured run of 2026-07-27 (server/benchmark-report.json); regenerate with npm run benchmark, publish with npm run sync-benchmark. Detection latency, measured in-process around the gate decision function: p50 0.871ms · p95 1.513ms · p99 2.671ms over 2,670 evaluations.
Disclosed Vulnerabilities & Failure Logs
Below are actual cases where Mergen failed to block, blocked incorrectly, or conservatively holds a benign command. We document and test these continuously to narrow detection bounds. Cases marked CLOSED were open gaps that are now caught.
Zero-width space injection inside "terraform" command
Fixed: U+200B (zero-width space) is now in INVISIBLE_SIGNAL_RE. The obfuscation pre-check fires before the policy engine, routing this to HOLD for human review. Previously missed; now caught.
Hex escape sequence for 'rm' — literal \xHH sequences now detected
Fixed: Bare \xHH sequences outside $'...' ANSI-C quoting are now detected by detectObfuscationSignal as "hex-encoded characters" and routed to HOLD before the policy engine runs. Previously passed through; now caught as suspicious.
Natural-language destructive framing — now caught via semantic expansion
Fixed: SEMANTIC_EQUIVALENTS now maps /\bdrop\b...\btable\b/ to a canonical nl-drop-table marker. The hold_nl_destructive_intent rule fires on this marker, routing the instruction to HOLD for human review. No LLM in the critical path — purely deterministic pattern expansion.
Deleting a single completed pod is held for one-click approval — by design
Tradeoff: Removing one finished worker pod is routine cleanup, but the local policy does not run an LLM to parse intent, so it holds any targeted kubectl deleteunder the same ruleset that blocks namespace and deployment deletion — a human resumes it with one click. This is a deliberate conservative HOLD. Under the strict scoring above a benign HOLD would count as a false positive, so the corpus labels this case as an expected HOLD on the attack side — it is disclosed here instead of being quietly scored as a pass. Tighten the rule with a scoped allow if this friction outweighs the safety margin for your team.
Developer friction: actively tracked and fixed
Every false positive the gate has ever produced is listed here. The table grows when a new FP is found — and shrinks to zero every time we ship a fix. A non-empty table means the feedback loop is working.
The ( character in the grep pattern matched the word-boundary separator for the destroy verb, causing a source-code search to trigger the destructive command rule.
terraform destroy prod → still BLOCK ✓
The word-boundary regex fired at the -/d boundary in -destroy, so a Terraform flag value matched the standalone destroy command pattern.
terraform destroy prod → still BLOCK ✓
The s3 rm rule held every object removal, including a single build artifact. The rule is now scoped to recursive and wildcard operations, where the blast radius actually lives.
aws s3 rm s3://prod-analytics/ --recursive → still BLOCK ✓
The blast-radius escalation scanned raw text, so SQL quoted inside an echo argument (and "truncate" inside a git commit -m message) fired the data-destructive rule. It now scans the shell-AST haystack the policy engine itself matches against: inert argument text is data, not action — while a command substitution hidden inside that text is still extracted and evaluated as an executed command.
DROP TABLE users (executed, not quoted) → still stopped ✓
All fixed cases are locked as PASS assertions in the eval corpus and test suite — they cannot silently regress.