Architecture & Evaluation
How Mergen intercepts an agent's shell commands and file edits and reaches a deterministic verdict on your own machine (p50 3.1ms, measured), and the test harness that validates every policy claim.
Who governs what,
before a single call runs.
This is the authority chain: who writes a policy, and how it reaches a developer's machine. The interception diagram below tells a different, equally true story: how one call actually gets evaluated once that policy is already in place.
Platform Team
Writes policy: what every agent is and isn’t allowed to do.
My Organization
Publishes it from mergen.app. Every signed-in machine pulls the published policy on its own, every two minutes by default. Teams that keep everything in-house run the same sync on their own hardware.
Mergen Runtime
Every command, file edit and CLI call on that machine goes through the same local gate, whichever IDE sent it. p50 3.1ms over 2,755 measured evaluations.
Developer
Works inside the policy: sees PASS, HOLD, or BLOCK on every action, in the tools they already use.
One call, one gate,
one verdict.
Agent calls a tool
A shell command, or a file edit the IDE is about to write.
Pre-execution hook · self-heal watcher
Claude Code’s PreToolUse hook, Windsurf’s pre_write_code / pre_run_command and Cursor’s beforeShellExecution stop the call before it runs. Codex’s PreToolUse hook does the same for commands and patches once you trust it in Codex (/hooks). Cursor file edits and the plain VS Code extension use a detect-and-revert self-heal watcher instead, since neither exposes a pre-write hook for edits: the write briefly lands on disk before Mergen reverts it.
applyGate()
server/src/intelligence/tool-guard.ts: every shell command, whichever IDE or tool sent it. JSON policy rules are evaluated against the parsed command (its programs, arguments, targets and SQL), not its raw text. p50 3.1ms over 2,755 measured evaluations.
evaluateEditGate()
server/src/intelligence/edit-gate.ts: every file edit, judged on the pending diff before it lands. Deterministic detectors look for introduced secrets, removed auth checks, deleted or skipped tests, removed CI checks, edits to the agent's own hooks and settings, and writes to places review never sees (shell startup files, git hooks, outside the repo). A hardcoded secret blocks; the rest hold for a person. Unsafe settings, untrusted dependency sources, unsafe code and publicly exposed infrastructure are flagged, not stopped, unless your team turns one into a hold.
Original handler runs, unchanged.
Handler never runs. The agent gets the reason and a safer alternative.
Waits for a person to approve it in the editor, the CLI or Slack. Runs only once they do.
On Cursor file edits and the VS Code extension, BLOCK and HOLD both mean detect-and-revert: the edit has already landed, so Mergen undoes it rather than stopping it.
Enforcement happens where the action happens.
The agent asks to execute. Mergen evaluates locally: no cloud round-trip, no LLM in the loop, no dependency on being online. The action proceeds, waits for a human, or never reaches the handler.
551-Case Regression Harness
We do not claim 100% security. Mergen is evaluated against a labeled adversarial dataset (322 attacks, 229 benign) run through the exported gate decision function the live interceptor itself calls, not a test-only mirror. Our results disclose exact failure categories, open evasion gaps, and known false positives so you can calibrate policy thresholds according to your risk tolerance.
Breakdown covers the 322 attack fixtures. The 229 benign fixtures are scored separately (0 flagged) under a strict definition: any verdict other than PASS on a benign command, including a conservative HOLD, counts against us. That score covers the policy layer. The live gate adds stateful layers a corpus cannot carry (script trust, session contamination, override corpus, agent scrutiny, risk score), and because those can only escalate PASS toward HOLD, both headline numbers are floors: live detection is never below the measured figure, and neither is the live false-positive rate. Of the 229 benign
fixtures, 4 invoke an interpreter on a script file and
0 use a heredoc body, the two shapes that produce most real-world
interruption. Numbers are imported from the measured run of 2026-10-14. Detection latency, measured in-process around the gate decision function: p50 3.081ms · p95 5.845ms · p99 13.644ms over 2,755 evaluations. That is the gate's own decision, not what a session feels: on Claude Code and Windsurf each tool call also starts the hook process and makes one local round trip, about 300ms end to end on a warm macOS install, almost all of it process startup.
Disclosed Vulnerabilities & Failure Logs
Below are actual cases where Mergen failed to block, blocked incorrectly, or conservatively holds a benign command. We document and test these continuously to narrow detection bounds. Cases marked CLOSED were open gaps that are now caught.
Zero-width space injection inside "terraform" command
Fixed: U+200B (zero-width space) is now in INVISIBLE_SIGNAL_RE. The obfuscation pre-check fires before the policy engine, routing this to HOLD for human review. Previously missed; now caught.
Hex escape sequence for 'rm': literal \xHH sequences now detected
Fixed: Bare \xHH sequences outside $'...' ANSI-C quoting are now detected by detectObfuscationSignal as "hex-encoded characters" and routed to HOLD before the policy engine runs. Previously passed through; now caught as suspicious.
Natural-language destructive framing, now caught via semantic expansion
Fixed: SEMANTIC_EQUIVALENTS now maps /\bdrop\b...\btable\b/ to a canonical nl-drop-table marker. The hold_nl_destructive_intent rule fires on this marker, routing the instruction to HOLD for human review. No LLM in the critical path: purely deterministic pattern expansion.
Deleting a single completed pod is held for one-click approval, by design
Tradeoff: Removing one finished worker pod is routine cleanup, but the local policy does not run an LLM to parse intent, so it holds any targeted kubectl deleteunder the same ruleset that blocks namespace and deployment deletion; a human resumes it with one click. This is a deliberate conservative HOLD. Under the strict scoring above a benign HOLD would count as a false positive, so the corpus labels this case as an expected HOLD on the attack side. It is disclosed here instead of being quietly scored as a pass. Tighten the rule with a scoped allow if this friction outweighs the safety margin for your team.
Developer friction: actively tracked and fixed
Every false positive this corpus has caught is listed here, and each one stays as a permanent regression case after it is fixed. The corpus runs through the live gate in a model repository, so it is not a census of every false positive the product has ever produced; interruptions on real machines (an unreviewed script waiting for approval, for example) are measured separately, against real recorded outcomes.
The ( character in the grep pattern matched the word-boundary separator for the destroy verb, causing a source-code search to trigger the destructive command rule.
terraform destroy -auto-approve prod → still BLOCK ✓
The word-boundary regex fired at the -/d boundary in -destroy, so a Terraform flag value matched the standalone destroy command pattern and was hard-blocked. apply -destroy is a teardown, though, so it is now read as one: without -auto-approve it waits for a person, exactly like terraform destroy.
terraform destroy -auto-approve prod → still BLOCK ✓
The s3 rm rule held every object removal, including a single build artifact. The rule is now scoped to recursive and wildcard operations, where the blast radius actually lives.
aws s3 rm s3://prod-analytics/ --recursive → still BLOCK ✓
The blast-radius escalation scanned raw text, so SQL quoted inside an echo argument (and "truncate" inside a git commit -m message) fired the data-destructive rule. It now scans the shell-AST haystack the policy engine itself matches against: inert argument text is data, not action, while a command substitution hidden inside that text is still extracted and evaluated as an executed command.
DROP TABLE users (executed, not quoted) → still stopped ✓
All fixed cases are locked as PASS assertions in the eval corpus and test suite; they cannot silently regress.