HITL Setup: Human-in-the-Loop Approval Workflow

Mergen's HITL (Human-in-the-Loop) system suspends sensitive AI agent tool calls and routes them to a Slack thread for human approval before execution resumes. This is the feature that Humanloop was building before being acquired by Anthropic; Mergen ships it today.

How it works

AI agent calls a sensitive tool
    │
    ▼
Mergen gate evaluates → verdict: HOLD
    │
    ├── Promise suspended (handler NOT called yet)
    ├── Slack message fired to your #mergen-approvals channel
    │   "Agent wants to: kubectl delete namespace production
    │    Why: Targeted data mutation by AI agent
    │    [✅ Approve] [❌ Deny]"
    │
    ▼
Human clicks Approve or Deny in Slack
    │
    ├── Approve → handler executes, result returned to agent
    └── Deny    → agent receives cancellation error, reformulates

If no human responds within the timeout (default: 10 minutes), the hold is automatically denied and logged as a timeout in the Agent Blunder Log.

Step 1: Create a Slack bot

1. Go to api.slack.com/apps → Create New App → From Scratch
2. App name: Mergen | Workspace: your workspace
3. OAuth & Permissions → Bot Token Scopes:
   • chat:write
   • channels:read
   • channels:history (for MERGEN_SLACK_OVERRIDE_LOOP)
4. Install to Workspace → copy the Bot User OAuth Token (xoxb-...)
5. Basic Information → Signing Secret → copy it
6. Invite the bot to your approvals channel: /invite @Mergen

Step 2: Configure Mergen

Add to your environment (or server/.env):

MERGEN_SLACK_BOT_TOKEN=xoxb-your-token-here
MERGEN_SLACK_CHANNEL=#mergen-approvals
MERGEN_SLACK_SIGNING_SECRET=your-signing-secret-here
MERGEN_PUBLIC_URL=https://mergen.your-domain.com  # required for approve/deny links

MERGEN_PUBLIC_URL is required. Slack url: buttons open $MERGEN_PUBLIC_URL/hitl/approve?token=<uuid> in the user's browser. Without it, buttons won't work.

Restart Mergen: mergen-server start

Step 3: Verify the HITL webhook

Test by triggering a HOLD manually:

# Simulate a sensitive tool call via the gate
curl -X POST http://127.0.0.1:3000/gate/evaluate \
  -H 'Content-Type: application/json' \
  -H 'x-mergen-secret: your-secret' \
  -d '{"command": "kubectl delete namespace staging", "toolName": "bash"}'

You should see a Slack message appear within 2 seconds. Click Approve or Deny in Slack.

Step 4: Check the pending queue

# List all currently held tool calls
curl http://127.0.0.1:3000/hitl/pending \
  -H 'x-mergen-secret: your-secret'

Response:

{
  "ok": true,
  "count": 1,
  "pending": [{
    "token": "abc123",
    "toolName": "bash",
    "commandArg": "kubectl delete namespace staging",
    "policyReason": "Targeted data mutation or resource removal by AI agent.",
    "suggestedAlt": "Run kubectl get namespace staging first to confirm state, then use kubectl delete with --dry-run=client to preview.",
    "triggeredRules": ["hold_agent_data_mutations"],
    "heldAt": 1720269600000,
    "waitingSec": 45,
    "quorumRequired": 1,
    "approvedCount": 0,
    "approveUrl": "POST /hitl/approve?token=abc123",
    "denyUrl": "POST /hitl/deny?token=abc123"
  }]
}

Step 5: Programmatic approve/deny (CI / scripts)

# Approve
curl -X POST "http://127.0.0.1:3000/hitl/approve?token=TOKEN" \
  -H 'x-mergen-secret: your-secret'

# Deny
curl -X POST "http://127.0.0.1:3000/hitl/deny?token=TOKEN" \
  -H 'x-mergen-secret: your-secret'

# Approve with constraints (agent must honour these params)
curl -X POST "http://127.0.0.1:3000/hitl/approve-constrained?token=TOKEN" \
  -H 'Content-Type: application/json' \
  -H 'x-mergen-secret: your-secret' \
  -d '{"constraints": {"targetNamespace": "staging", "dryRun": true}}'

What triggers a HOLD

By default, the following trigger a HOLD (not a BLOCK) for AI actor tool calls:

To convert a HOLD to a BLOCK (no human approval needed, just deny): add the pattern to a block_* rule in your policy. See mergen-server policy-push.

HITL approval fatigue

If your team is receiving too many HITL notifications, check:

curl http://127.0.0.1:3000/hitl/fatigue -H 'x-mergen-secret: your-secret'

The fatigue endpoint reports whether the hold rate has exceeded the threshold in the last hour and suggests converting noisy HOLD rules to BLOCK rules so humans only see the genuinely ambiguous cases.

RBAC: Who can approve

If your team has multiple on-call engineers, configure RBAC so only people with the responder role can approve HITL holds:

# Add a member with responder role
curl -X POST http://127.0.0.1:3000/rbac/members \
  -H 'Content-Type: application/json' \
  -H 'x-mergen-secret: your-secret' \
  -d '{"memberId": "alice@company.com", "role": "responder"}'

Once any member is configured, approvals require the x-mergen-member header to identify the approver. Anonymous approvals are rejected.

Quorum approvals

For high-risk operations, require two approvers:

{
  "name": "hold_prod_delete_quorum",
  "type": "warn",
  "patterns": ["kubectl delete namespace production"],
  "quorumRequired": 2,
  "reason": "Deleting production namespace requires two approvals."
}

Add via POST /policies and mergen-server policy-pull.

Timeout configuration

A hold nobody answers is denied after 15 minutes. The window is fixed; there is no setting to change it.

Timed-out holds are denied and logged; the command never runs. The agent is told the approval window expired and must ask again.

A mergen-server restart (crash, redeploy, mergen-server stop/start, doctor --fix) never silently runs or drops a pending hold. A hold's resolver lives in the server process, so it can't survive a restart. On the next start:

• an unexpired shell command hold is checked again against the current policy. If it still needs a person, it is held again and waits for your answer; the agent's retry joins it. Nothing runs on the replay itself.
• an edit hold, an in-session (inline) prompt hold, and any expired hold are denied and logged. The agent retries the original tool call, the same recovery as after a BLOCK.

The override corpus loop

When you approve a HITL hold with "Remember for this service", Mergen encodes the approval as an override corpus entry. The next time the same agent action is attempted in the same context, Mergen checks the corpus and can escalate autonomously without paging the on-call, or hold again if the context has changed.

See docs/OVERRIDE_CORPUS.md for the full override corpus documentation.

Troubleshooting

# Confirm HITL is wired correctly
mergen-server doctor