agent-seatbelt

The problem it answers

You run AI agents with real credentials against real systems. The README describes an independent evaluation of one leading runtime in which the blocking held up, but its audit trail had holes: a blocked DNS query left no event at all, the audit export silently wrote nothing, and tool calls inside the sandbox were never recorded. When something goes wrong you cannot answer the only question that matters — what exactly did the agent do, and what did we stop? — from records you can trust, because the records are scattered across the runtime, the gateway, and a console, and some of them can be edited after the fact.

Seatbelt is the missing layer, in the README's words: one policy point and one evidence ledger across runtimes, with coverage that is checked, not claimed. Its core rule: an agent action with no evidence record is a system failure, not a detail — the ledger reconciles what was expected against what was observed, and a run fails publicly when a record is missing.

What it does

• Deterministic policy at the boundary. Tool calls, egress, DNS, file access, budget, and credential grants are decided by OPA/Rego policy packs running outside the agent process. No LLM judges anything in the decision path.
• Records for both directions. Allow and deny become evidence records carrying the enforcement mode and the hash of the exact policy pack that decided them.
• Tamper-evident ledger. Append-only NDJSON with a hash chain, signed checkpoints, and Merkle roots anchored outside the ledger directory. Flip one byte and the verifier names the exact record.
• Coverage accounting. Each adapter publishes a coverage manifest. After a run, the accountant reconciles the ledger against an independent ground truth; a missing record fails the run in writing.
• Credentials per run + per action. A broker grants credential use for one run, one action, one target. The model never sees the raw secret; the ledger records the grant or the refusal, never a value. Denied DNS names are recorded redacted (digest only) so the ledger can never become a copy of what it blocked.
• One-command offline verification. Chain integrity, checkpoint signatures, external anchor, coverage report, and a raw-secret scan of the whole bundle — no server, no trust in the runtime.

Adapters (phase 1)

Each adapter publishes a coverage manifest in the repository (adapters/claude-code/coverage-manifest.json and adapters/openshell/coverage-manifest.json).

Use it

The README gives no npm package and no install command — it is run from source. Prerequisites: Python ≥ 3.11 and an OPA binary (phase 1 developed against OPA 1.21); everything else is the Python standard library. The golden path takes about 2 minutes:

export PYTHONPATH="$PWD/src"
export OPA_BIN=/path/to/opa

# A FAKE demo credential for the sample workload. Never a real one:
# the broker reads it from the environment, and it must never reach
# the agent or the ledger.
export SEATBELT_SECRET_BILLING_API='DEMO-SECRET-billing-9c1f'

D=/tmp/seatbelt-demo; mkdir -p "$D/run" "$D/anchors" "$D/ws" "$D/bundle"

# 1. Start the sample target API and the Seatbelt daemon
python3 examples/ops-assistant/target_server.py 18741 &
python3 -m seatbelt.cli serve --run-dir "$D/run" --run-id demo-1 \
  --adapter openshell --workspace "$D/ws" --policies policies \
  --opa-bin "$OPA_BIN" --key-file "$D/operator-key.txt" \
  --anchor-dir "$D/anchors" --state-file "$D/state.json" &
sleep 2

# 2. Run the clean sample workload through Seatbelt's boundary points
python3 examples/ops-assistant/agent.py --mode openshell \
  --scenario clean --state-file "$D/state.json" --workspace "$D/ws" \
  --ground-truth "$D/ground-truth.ndjson" \
  --transcript "$D/transcript.json" --target-port 18741

# 3. Stop the daemon, reconcile coverage, assemble + verify the bundle
kill %2; sleep 1
python3 -m seatbelt.cli coverage --run-dir "$D/run" \
  --ground-truth "$D/ground-truth.ndjson" --key-file "$D/operator-key.txt" \
  --adapter openshell --anchor-path "$D/anchors/anchors.ndjson"

cp "$D/run/ledger.ndjson" "$D/run/coverage-report.json" \
   "$D/anchors/anchors.ndjson" "$D/bundle/"

printf '%s\n' "$SEATBELT_SECRET_BILLING_API" 'CANARY-SECRET-7f3a9d2e' \
  > "$D/secrets.txt"

python3 -m seatbelt.cli verify --bundle "$D/bundle" \
  --key-file "$D/operator-key.txt" --secrets-file "$D/secrets.txt" \
  --ground-truth "$D/ground-truth.ndjson"

Commands copied exactly from the README quickstart. The verifier prints {"ok": true, ...} and exits 0. Try the planted scenario to watch five planted violations get blocked and recorded, then flip any byte in bundle/ledger.ndjson and re-run the verify step — it fails, naming the tampered record. The demo is one real terminal session, recorded as it ran: bash docs/demo/run_demo.sh runs the clean workload, then the planted workload, reconciles coverage, verifies the evidence bundle offline, then flips one byte in a copy of the ledger and watches the verifier fail, naming the tampered record. Raw session text: docs/demo/typescript.txt in the repo.

Measured numbers (README)

• Phase-1 acceptance: the hook CLI, MCP proxy, egress proxy, DNS tap, broker, ledger, accountant, and verifier are measured end to end by verification/run_acceptance.py — all checks pass (full results in verification/results.md).
• Live OpenShell 0.1.5 run: on a GitHub Actions runner (docker driver) the sandbox booted and executed real commands; a native shorthand event for a DNS denial was captured (NET:REFUSE … DENIED, reason policy_dns_ineligible); Seatbelt-side coverage passed 13/13 against independent ground truth (workflow .github/workflows/openshell-live.yml; full record in the release dossier).
• Measured hole, stated in the README: with ocsf_json_enabled set exactly as the vendor documents, the runtime's OCSF-JSON export produced zero records in both live runs, so the native JSON stream still has nothing to ingest.
• Demo: clean workload (every action allowed and recorded) + planted workload (five planted violations blocked and recorded) + offline bundle verification + the 1-byte tamper case failing verification.
• Test suite: unit + component + concurrency tests — 23 tests (per the README layout).

The README publishes no latency, throughput, or benchmark timing numbers, so none are shown here.

What this is NOT — and where rivals beat us

• Not a sandbox. Seatbelt provides no isolation — by design, and never will. If you need a sandbox, run one of the rivals' runtimes under Seatbelt. OpenShell and nono provide the actual isolation; Seatbelt does not.
• nono shipped hash-chained agent audit before Seatbelt (inside its sandbox). Infisical Agent Vault is a polished commercial product with support. Seatbelt's claim is narrower: unifying policy, evidence, credential grants, and verified coverage on one neutral layer.
• Comparison, as stated in the README: Seatbelt is a policy + evidence layer above runtimes (runtime-neutral: one ledger across runtimes; records denied actions by contract with coverage checks; credential grants per run + per action in the same ledger). OpenShell is a sandbox runtime (not runtime-neutral; a shorthand event exists natively, but its documented OCSF-JSON export measured 0 records on 0.1.5, so nothing reaches an outside ledger; no equivalent published coverage verification; credential injection at egress inside its sandbox). nono is a sandbox with tamper-evident audit (not runtime-neutral; records denied actions inside its sandbox; no equivalent published coverage verification). Infisical Agent Vault is a credential broker, commercial, Preview (partially runtime-neutral — HTTP(S) proxy only; audit log for brokered calls; no coverage verification; grants per service bundle).

Limits (stated in the README)

• The OCSF hole above is the runtime's, not a reconstruction gap: the documented supervisor file sink sits on a tmpfs that the vendor's own docs say external tools cannot read. Closing that hole is the boundary recorder's reason to exist.
• The Claude Code hook path is exercised through the real hook protocol (JSON on stdin, decision JSON on stdout) via subprocess; a live interactive Claude Code session is still pending the owner's Anthropic authentication.
• Phase-1 scope is the two adapters above. Phase 2 (README roadmap): an E2B adapter; sealing exports from observability tools into verifiable evidence; broker coverage for database/SSH patterns; a coverage re-measurement of OpenShell on its remaining Linux drivers (the docker driver was measured live in phase 1), published; richer alerting. Phase 3: more runtimes, public transparency anchoring, CI mode as an agent-testing gate.

Repository layout (README): src/seatbelt/ (daemon, PDP client, ledger, broker, proxies, verifier); policies/ (Rego policy pack v1 + data); adapters/ (claude-code/ and openshell/ adapters + manifests); examples/ (ops-assistant sample workload, clean + planted); verification/ (phase-1 acceptance harness and measured results); docs/ (architecture, threat model, demo); tests/ (23 tests). Further reading in the repo: architecture and trust model, policy pack v1, both adapter READMEs, sample workload, phase-1 acceptance results, release dossier, security policy.

License

Apache-2.0 — see LICENSE in the repository.

Source + README on GitHub → · Report an issue → · ask@techdev.fun

Related tools

More open-source tools from TECHDEV: sessguard, runverdict, gheist-guard, vetgate, context-diet, ci-failure-detective, devsecops-os, env-doctor, skill-audit, logtrim, testguard, breakcheck, keymap, memgraph, ggufit, leaseguard, factfence, jsonfold

Have a say

Vote for this tool, suggest a feature, or discuss it with other operators on the community board.