Hardening the Veltro suite

Source of truth: docs/hardening.md (synced at build time — edit the original, not this copy). Operational companion to docs/security/threat-model.md: what the shipped assembly exposes, how to shrink it, how to rotate the secrets it holds, how backups are encrypted, and which TLS profile to run. Everything here describes code and configuration that exists in this repository — where a control is the operator’s responsibility rather than the suite’s, it says so.

1. Network exposure matrix

These are the ports the shipped Compose assembly publishes. Everything not listed is reachable only on the internal Compose network.

Service (profile)	Published	Default bind	Purpose	Hardening
`veltro-nginx` (`core`)	`${VELTRO_HTTP_PORT:-80}` → 80	all interfaces	The single public origin: identity at `/`, installed products under `/vectorflow`, `/sigmatic`, `/warden`	Put it behind the `tls` profile (§4) or your own terminating proxy; never expose plain HTTP off-host
`veltro-tls` (`tls`)	the `PUBLIC_BASE_URL` port (443 by default), 80, 8443	all interfaces	TLS front door (Caddy) and TLS log ingest	The only ports that should face an untrusted network
`sigmatic-backend` (`sigmatic`)	8001	`${SIGMATIC_INGEST_BIND_ADDRESS:-127.0.0.1}`	Direct log ingest for shippers that cannot go through the gateway	Keep the default loopback bind, or an RFC1918/ULA address. Sigmatic’s startup preflight refuses a public or unspecified bind — do not work around it; use the `tls` profile’s 8443 site for ingest that must cross a network
`prometheus`, `alertmanager`, `grafana` (`monitoring`)	9090, 9093, 3001	`127.0.0.1`	Platform self-observability	Already loopback-only; reach them over SSH port-forwarding rather than republishing
`oidc-mock`, `saml-mock`, `misp-mock`, `crowdsec-mock`, `crowdsec-cscli` (no published port)	high ports	`127.0.0.1`	CI fixtures, dispatch-only	Never enable on a production host

Rules that follow from the matrix:

One origin. Only the gateway (or the TLS front door) needs to be reachable. No product publishes a UI or API port of its own; every product is served through the gateway behind Veltro identity.
Bind narrowly. Compose port mappings accept a host_ip prefix (127.0.0.1:8080:80); use it whenever a port exists only for local debugging.
Egress is opt-in. The suite makes no outbound call you have not configured: integration connections, threat-intel feeds, the AI provider, and the CLI’s release-check against the GitHub releases API. Restricting egress to those destinations does not break the suite.
Agents dial in, not out. VectorFlow agents connect to the control plane with an enrolment token; the control plane never needs inbound access to an agent host.
Verify what you exposed. veltro doctor reports listening ports and missing required keys; veltro status probes each component’s health endpoint.

2. Secrets

2.1 Where they live

scripts/init-env.sh generates the deployment’s secrets into .env on first install; the shipped .env.example carries change-me-* placeholders so a half-configured deployment is obvious. Identity’s TOTP keyring lives in the identity-keyring volume: identity-init creates it at mode 0600 and identity mounts it read-only. Backups seal it under VELTRO_BACKUP_PASSPHRASE.

Never put a secret in a shell argument, a URL query string, an image label, or a log line: those surfaces end up in docker inspect, in the run log, and in a support bundle. The suite’s own code reads secrets from the environment or from a mounted file for exactly this reason, and veltro support-bundle redacts.

2.2 Rotation runbook

Rotation is per-secret because the blast radius differs. In every case: change the value in .env (or the secret backend), restart the services that read it, then verify with the stated check. Rotate on a schedule and immediately after any suspected exposure.

Secret	Rotate by	Restart	Verify
`VELTRO_SESSION_SECRET`	New random value in `.env`	Identity plus every product that verifies sessions	Existing sessions are invalidated: a logged-in browser is sent back to `/login`. Rotating this is the fastest way to sign every analyst out
Identity signing key ring (`VELTRO_IDENTITY_SIGNING_PASSPHRASE`, ring rows)	`signing-tool rotate`, per `docs/runbooks/identity-signing-key-provisioning.md`. The old key stays in identity’s JWKS as verify-only, and products pick the new one up from the JWKS	Identity only	`/api/health/ready` healthy, `/.well-known/jwks.json` lists the new key id, and a fresh login reaches all three products
Identity database role passwords (`VELTRO_IDENTITY_{OWNER,RUNTIME,IMPORTER,RECOVERY}_PASSWORD`)	`ALTER ROLE … PASSWORD` in `suite-postgres`, then the `.env` value (the password inside each identity DSN too)	Identity, migrator	`veltro status` shows identity ready; `identity-init` exits 0
Product database passwords (`VF_POSTGRES_PASSWORD`, `SIGMATIC_POSTGRES_PASSWORD`, `WARDEN_POSTGRES_PASSWORD`, `OPENSEARCH_ADMIN_PASSWORD`)	`ALTER ROLE` in `suite-postgres`/OpenSearch security API, then `.env`	That product’s services (Sigmatic also `sigmatic-pgbouncer`)	Product health endpoint returns ready
Product JWT/session/encryption keys (`SIGMATIC_JWT_SECRET_KEY`, `SIGMATIC_SESSION_SECRET_KEY`, `SIGMATIC_ENCRYPTION_KEY`, `WARDEN_JWT_SECRET_KEY`, `WARDEN_SESSION_SECRET_KEY`, `WARDEN_SECRET_KEY`, `VF_ENCRYPTION_KEY_V2`)	New value in `.env`. Encryption keys are not a free rotation: stored ciphertext is bound to the key, so follow the product’s key-rotation path (re-encrypt) rather than swapping the value under live data	That product	Product boots, and a stored connection credential still decrypts (open a connection’s test action)
Agent enrolment tokens (`VF_ENROLLMENT_TOKEN`) and node tokens	Rotate in VectorFlow’s environment settings; re-enrol with `veltro agent enroll`	Agents	The fleet page shows the node healthy under the new token; the old token is refused
Sigmatic ingest tokens (per data source)	Rotate on the data source; update the shipper	Shipper only	Events keep arriving; the old token is refused
`VF_METRICS_TOKEN`, `SIGMATIC_METRICS_TOKEN`, `WARDEN_METRICS_TOKEN`	New value in `.env`	That backend and Prometheus	The scrape target returns to `UP`
`WARDEN_WEBHOOK_SECRET` and per-source webhook secrets	Rotate on the source, update the sender	Warden backend	A test delivery is accepted; an unsigned delivery is refused and audited
Integration and response-adapter credentials (firewall, EDR, cloud, mail, ticketing)	Rotate in the upstream system, then update the Warden/VectorFlow connection	None	The connection’s test action succeeds; an action against the old credential fails
AI provider API key	Rotate with the provider, update the product’s AI settings	None	The provider test call succeeds; an invocation appears in the audit trail
`GRAFANA_ADMIN_PASSWORD`	New value in `.env`, then change it in the application	That service	Login with the new credential
`VELTRO_BACKUP_PASSPHRASE`	New value in `.env`. Keep the old passphrase for as long as you keep archives sealed with it — an archive cannot be re-sealed retroactively	Backup sidecar	The next backup opens with the new passphrase (`veltro restore --dry-run`)

Where a secret backend is configured (Vault, AWS/GCP/Azure secret managers), rotate in the backend and let the product resolve the new value at delivery time; the reference in the configuration does not change.

After any rotation that invalidates credentials, check that the refusals were audited — a rotation that produces silent failures is indistinguishable from an attack in progress.

3. Backups and their encryption

veltro backup produces a veltro-backup/v1 archive containing pg_dump -Fc dumps of the identity, VectorFlow, Sigmatic and Warden databases, an OpenSearch snapshot of the sigmatic-* indices, Redis dumps, the named data volumes, and a manifest.json binding every artifact to a sha256 digest.

What is protected, and how:

Integrity. Restore verifies every artifact digest against the manifest and refuses a mismatch, refuses a suite-major or module-version regression (restore then upgrade, never the reverse), and refuses an identity schema regression against conformance/identity-persistence-v1.json.
Secret material. The identity key ring and bootstrap secret files are sealed inside the archive as a veltro-secret-envelope/v1 — AES-256-CBC with a PBKDF2-SHA256-derived key, from VELTRO_BACKUP_PASSPHRASE. Without the passphrase the sealed member cannot be opened, and restore says so instead of silently continuing.
.env is excluded by default. --include-env opts in and prints a warning; the manifest records the sha256 of .env either way, so restore can tell you the deployment configuration has changed.
The rest of the archive is not encrypted. Database dumps contain your telemetry, cases and stored (product-encrypted) credentials. Treat the archive as sensitive: write it to an encrypted volume or object store with server-side encryption, restrict access to the operator role, and set BACKUP_KEEP so old copies do not accumulate beyond your retention policy.

Practical setup:

veltro backup --schedule "0 3 * * *"   # profile `backup` sidecar, BACKUP_KEEP=7
veltro restore /path/to/veltro-backup-<ts>.tar --dry-run

Test the restore, not the backup: an untested archive is a hope, not a control. The dr job in verify.yml exercises seed → backup → wipe → restore → golden-path on the homelab, and the same sequence is what you should rehearse. See docs/runbooks/backup-restore.md for the step-by-step operator procedure.

4. TLS profiles

The recommended setup is your own reverse proxy or load balancer terminating TLS in front of the gateway. That’s the installer’s default (--tls off), and the docs site’s “TLS and your reverse proxy” page (operate/tls) lists what the proxy must forward, with Caddy and nginx examples. For an install with no proxy, compose/tls.yml (profile tls) runs Caddy in front of the suite. It’s selected at install time with veltro install --tls <mode>, which writes compose/Caddyfile and sets VELTRO_TLS_MODE and PUBLIC_BASE_URL.

Mode	Certificate source	Use when	Notes
`off` (default)	your proxy	Your own proxy terminates TLS (recommended)	The gateway serves plain HTTP on `VELTRO_HTTP_PORT`; that port must not leave the host or the private network
`internal`	Caddy’s internal CA	LAN or lab deployments	The CA certificate must be trusted on each analyst machine; the installer prints its path
`custom`	`tls/fullchain.pem` + `tls/privkey.pem`	You already have a certificate (corporate CA, wildcard)	Mounted read-only; renewal is your process

Additional rules:

When Sigmatic is installed, the CLI-generated Caddyfile adds a second site on 8443 that proxies sigmatic-backend:8001, so log shippers that must cross a network get TLS ingest while the raw ingest port stays on its loopback/private bind. Without Sigmatic there is no ingest site.
Caddy listens on the port in PUBLIC_BASE_URL (443 when there isn’t one) and Compose publishes that same port. Set VELTRO_HTTPS_PORT only when something in front, like NAT, forwards a different host port. The URL can’t use 80 (Caddy’s HTTP redirect), 2019 (its health endpoint) or 8443 (the TLS ingest port, published whether or not Sigmatic is installed).
On a --tls internal install, veltro status and veltro demo seed trust the install’s own root (tls/caddy-data/caddy/pki/authorities/local/root.crt) alongside the system store. Other machines still need it trusted by hand.
PUBLIC_BASE_URL is load-bearing beyond cosmetics: single-origin session cookies, the gateway’s assertion mint and every product deep link are derived from it. Changing it is a reconfiguration, not a redirect — re-run veltro install --public-url … so the Caddyfile and product base URLs move together.
Identity’s cookies follow the scheme of PUBLIC_BASE_URL, not the request. With https:// its session, CSRF and SSO cookies are Secure (CSRF as __Host-veltro.csrf-token, the session as __Secure-authjs.session-token), so browsers must reach the suite over HTTPS. With http:// they stay plain, which a --tls off install with no proxy needs for sign-in to work.
Keep the certificate’s expiry monitored. The monitoring profile’s alert rules include certificate expiry; without that profile, monitor it yourself.

5. Verifying the artifacts you run

scripts/suite_signatures.py:PUBLISHERS is the authoritative repository-to-workflow map. Select the exact certificate-subject pattern for the artifact being checked; a union across repositories or workflows is intentionally not accepted.

vectorflow-server-suite, vectorflow-agent: ^https://github\.com/veltrosecurity/vectorflow/\.github/workflows/ci\.yml@refs/(?:heads/main|tags/v[^\s]+)$
sigmatic-backend, sigmatic-frontend: ^https://github\.com/veltrosecurity/sigmatic/\.github/workflows/docker-build\.yml@refs/(?:heads/main|tags/v[^\s]+)$
warden-backend, warden-frontend: ^https://github\.com/veltrosecurity/warden/\.github/workflows/docker-build\.yml@refs/(?:heads/main|tags/v[^\s]+)$
veltro-identity: ^https://github\.com/veltrosecurity/veltro/\.github/workflows/(?:release-suite|verify)\.yml@refs/heads/main$
identity (identity’s own release line, X.Y.Z and X.Y): ^https://github\.com/veltrosecurity/veltro/\.github/workflows/release-identity\.yml@refs/heads/main$
charts/vectorflow-server, charts/vectorflow-agent: ^https://github\.com/veltrosecurity/vectorflow/\.github/workflows/ci\.yml@refs/tags/v[^\s]+$
charts/sigmatic: ^https://github\.com/veltrosecurity/sigmatic/\.github/workflows/release\.yml@refs/tags/v[^\s]+$
charts/warden: ^https://github\.com/veltrosecurity/warden/\.github/workflows/release\.yml@refs/tags/v[^\s]+$

# Replace the pattern and artifact with one row from the table.
PUBLISHER_PATTERN='<exact certificate identity regexp>'
cosign verify \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  --certificate-identity-regexp "$PUBLISHER_PATTERN" \
  ghcr.io/veltrosecurity/<artifact>@sha256:<digest>

# Re-verify a release manifest: digests, contracts and recorded signatures.
python3 scripts/suite_manifest.py verify dist/veltro-suite-<version>.json

suite_manifest.py verify refuses a manifest whose recorded signature is not the one cosign accepts for the pinned digest, so a registry-side signature swap fails even when the replacement signature is itself valid. Run it against the manifest attached to the release you deployed, not against a manifest you regenerated.

6. Checklist

Only the TLS front door (or your own proxy) is reachable off-host.
SIGMATIC_INGEST_BIND_ADDRESS is loopback or a private address.
No change-me-* value remains in .env (veltro doctor checks).
TLS mode chosen deliberately; certificate expiry monitored.
VELTRO_BACKUP_PASSPHRASE set, archives stored encrypted, a restore rehearsed.
Rotation owner and schedule recorded for every secret in §2.2.
Monitoring profile enabled, or its alerts reproduced in your own stack.
Deployed image digests verified against the release manifest.
Analyst grants reviewed: default deny means an unused grant is the only way a stale account keeps access.