AI in Warden
Source of truth:
warden/docs/ai-governance.md(synced at build time — edit the original, not this copy). Warden’s AI features stay off until someone turns AI on in Veltro’s Settings → AI. Turning it off there stops them immediately. AI never closes a case or bypasses an approval gate. This page covers Warden’s data flow and audit records; the Veltro repository holds the suite-wide policy. Warden runs behind Veltro identity.
Configure
Section titled “Configure”Provider, model, and key live in Veltro identity’s Settings → AI, the one AI form for the suite. Identity writes them to Warden through GET/PUT /api/peer/v1/settings/ai, which only accepts identity’s own machine token (settings:read to read, settings:write to write, no actor). A write is either {"enabled": bool} alone or a full config: enabled, provider (openai_compatible, anthropic, or ollama), base_url, model, and an optional api_key. Both return enabled, provider, base_url, model, and key_configured. Each write is audited as ai.settings.updated, attributed to veltro-identity.
The API key is write-only: it is Fernet-encrypted into Warden’s own ai setting and never returned or logged. Omitting api_key on a later write keeps the stored key as long as provider and base_url stay the same. A write that changes either one needs a new key, so the old key is never sent to a different host.
Warden’s own Settings → AI page (GET/PUT /api/settings/ai, admin only) tunes how Warden uses the provider: allowed data classes (alerts, events, cases, pipelines), max_tokens, temperature, and timeout_seconds. It can’t change the provider or turn AI on or off.
GET /api/settings/ai/status is readable by any authenticated user so product surfaces can show the off state and deep-link to the settings page; it carries no secret and no configuration detail beyond provider and model.
An Ollama daemon you run yourself keeps prompts inside your own network. Warden ships no model and no Ollama container.
What is sent
Section titled “What is sent”| Feature | Data classes |
|---|---|
Provider connectivity probe (settings.test) |
none — a fixed two-line prompt |
Runbook draft (authoring.draft_runbook) |
none — the operator’s bounded description and optional base YAML |
Run explanation (authoring.explain_run) |
alerts, cases |
Bounded runbook agent (runbook.ai_agent) |
the data classes declared by the runbook step |
Every feature declares its data classes and is refused with ai_data_class_denied when the allowlist does not permit them. A Warden install whose allowlist is empty can prove connectivity and draft from operator-authored text, but cannot send a case or alert to a model.
Prompts are redacted before they are sent and before they are logged: bearer tokens, API keys, AWS access-key IDs, basic-auth credentials, private-key blocks, and connection-string passwords become a REDACTED marker. Provider error bodies are redacted before they are stored or returned.
Refusals are named
Section titled “Refusals are named”ai_kill_switch, ai_disabled, ai_not_configured, ai_invalid_config, ai_data_class_denied, ai_provider_error, ai_structured_output_invalid, ai_timeout. Gates are evaluated kill switch → enabled → provider configuration → data classes → request bounds and output schema.
The kill switch
Section titled “The kill switch”Veltro’s Settings → AI off switch writes {"enabled": false} to Warden, which stores it as the ai_kill_switch setting and the config’s enabled flag. While off, every AI feature refuses with ai_kill_switch, no request reaches a provider, each refusal is audited, and the provider configuration and key are kept, so turning AI back on restores the previous state.
Every provider call writes one ai.invocation row in the hash-chained audit log: product, feature, model, provider, sha256: digests of the redacted prompt and output, token counts, duration, and the acting analyst. Content is never stored. An invocation whose answer is rejected by a feature keeps its real token usage and carries the typed refusal_code.
An attempt that no provider received — kill switch, disabled, not configured, invalid config, denied data class — is audited as ai.refusal instead, carrying the typed refusal_code, a redacted message and the actor, with no digests and no token counts because nothing was sent.
AI never decides
Section titled “AI never decides”- An AI verdict on a case is a proposal with a confidence and a rationale. Disposition stays a human action; nothing auto-closes.
- Any write intent that originates from an AI or agent path becomes an approval request on the normal four-eyes path. An AI-suggested runbook still needs its trigger and its approvals.
- A drafted runbook may only use actions that exist in the registry and must pass
parse_and_validate; it is shown as a diff and saved only by a user action.
Measuring AI
Section titled “Measuring AI”The Weekly SOC metrics report and GET /api/metrics/soc expose this AI-spend shape for the selected window:
| Field | Meaning |
|---|---|
invocations |
Attempts that reached a provider |
refusals, refusals_by_code |
Refused attempts, by typed code |
prompt_tokens, completion_tokens, total_tokens |
Token totals as audited |
by_feature, by_model |
Top features and models by token count |
The values are derived directly from the audited ai.invocation and ai.refusal rows only for scheduled operator reports and global administrators. Other authenticated dashboard users receive a zeroed ai_spend object while their case and agreement metrics remain team- and private-case-scoped.
The audit action determines whether the provider was reached: an ai.invocation whose answer was later refused still contributes its real token usage, while an ai.refusal contributes none. Analyst agreement is computed from ai.triage verdicts and ai_triage_feedback, then exposed as SocMetrics.ai_triage_agreement.