ADR-014: Always-on core and additive suite products

Source of truth: docs/adr/014-suite-composition-and-packaging.md (synced at build time — edit the original, not this copy). Status: Accepted (2026-09-28). All five migration steps have shipped (#1063). ADR-016 amends it: the launcher is retired, and Decision 5’s single-product branding is extended to every surface.

Decision owner: Danny

Amends: ADR-013’s four-profile deployment matrix. Its Compose/Helm parity and single-CLI operator contract remain.

Supersedes on acceptance: ADR-010 §2’s standalone products with local identity and no runtime dependency on Veltro, and ADR-002 Decision 2’s requirement that a module boots on its own with its own login. The rest of both decisions stands, including each product’s fixed internal ownership boundary and product-native roles.

Context

The root docker-compose.yml has 88 lines and includes ten files (docker-compose.yml:1-49). Those eleven files contain 44 service definitions and 1,939 lines in total; the six additional files compose/{ws-proof,oidc-mock,saml-mock,ollama,assertion-config-proof,cutover-rehearsal}.yml bring the count to 2,617 lines. The 44 definitions include opt-in cutover, monitoring, backup, TLS, and agent services, not 44 containers in a normal installation (docker-compose.yml:23-47, compose/{cutover,monitoring,backup,tls,vectorflow}.yml).

With full and opensearch, the current stack starts 22 long-running containers and six one-shots. That counts two chad-worker replicas (compose/chad.yml:255-296): there are 27 selected service definitions, but 28 containers. It runs four PostgreSQL instances (identity-postgres, vf-postgres, chad-postgres behind chad-pgbouncer, warden-postgres), four Redis instances (vf-redis, chad-redis, chad-redis-queue, warden-redis), one OpenSearch, and three worker containers (two CHAD, one Warden). Both demo fixtures, misp-mock and crowdsec-mock, use full, so a normal make up starts them (compose/{veltro,vectorflow,chad,warden,misp-mock,crowdsec-mock}.yml; cli/veltro_cli/lib/compose.py:65-76).

The CLI’s installation profiles are exactly full, vectorflow, chad, and warden; Compose also has auxiliary profiles (cli/veltro_cli/lib/toml_config.py:12-27, docker-compose.yml:14-28). The CLI assigns VELTRO_INTEGRATED from profile == "full" and adds OpenSearch only for full (cli/veltro_cli/lib/compose.py:56-102). Installation, diagnostics, and status each branch on that profile (cli/veltro_cli/lib/install.py:237-240,284-295, cli/veltro_cli/lib/doctor.py:46-57, cli/veltro_cli/lib/status.py:89-101,134-143). The Helm validator rejects any partial product matrix, and the CLI’s Helm rendering has only those four rows (charts/veltro/templates/validate-profile.yaml:1-24, cli/veltro_cli/lib/k8s.py:12-45). There is no supported two-product suite shape.

The coupling is operational too. veltro-nginx waits for identity and all three product frontends (docker-compose.yml:67-78); identity sets all three VELTRO_PEER_*_URL values (compose/veltro.yml:20-28); and vectorflow-suite expects version values for all three products (compose/vectorflow.yml:49-58). A single public origin with /, /vectorflow, /chad/, and /warden/ remains required: the session cookie is host-only and nginx’s product routes use auth_request to mint browser assertions. VectorFlow’s suite image bakes in /vectorflow (docs/adr/001-topology-and-session.md:66-72, nginx/veltro.conf:1-30,40-75,83-167, compose/vectorflow.yml:128-160). A missing product cannot be handled by moving it to another origin.

The product repos have their own standalone Compose files. VectorFlow’s standalone stack has Postgres and no Redis service, whereas this suite starts vf-redis; their default image tags also lag the suite pins: CHAD 0.44.3 versus 0.45.0, Warden 0.12.3 versus 0.13.0, VectorFlow 0.19.3 versus 0.20.0 (../chad/docker-compose.yml:1-89,153-154,197-198, ../warden/docker-compose.yml:34-35,71-72,101-102, ../vectorflow/docker/server/docker-compose.yml:1-59, compose/vectorflow.yml:84-96, versions.env:95,169,206). The old topology diagram still labels identity as a VectorFlow image and connects it to vf-postgres; the service list and DB-isolation text repeat that obsolete sharing claim. Identity has its own identity-postgres (docs/discovery/RUNTIME-ARCHITECTURE.md:17,27-28,46-47,151-153, compose/veltro.yml:1-6,41-70). The vectorflow-suite dependency comment still says Veltro migrates the shared VF database (compose/vectorflow.yml:136-144).

The products were built one after another. VectorFlow came first, as a management plane for Vector fleets and pipelines that is useful well outside security. CHAD (detection) and Warden (playbooks and response) followed, and the suite was assembled around products that were already complete standalone applications. Each product still carries two ways to sign in: its own local login and SSO when standalone, and Veltro’s browser assertions in the suite. On 2026-09-26, suite or integrated-mode markers appeared in 27 tracked files in VectorFlow, 41 in CHAD, and 44 in Warden, and each product had drifted from the suite contract in a different way (see Module contract).

Peer survey

These are installation shapes in the cited first-party docs and configuration, not endorsements. A shared datastore below means shared by a peer’s own components, not a database shared across vendors. The survey does not establish that another project has Veltro’s cookie or path-prefix requirements.

Peer	Packaging and optional modules	Shared datastores	Gateway and URL layout	Installing a subset; primary sources
GitLab Omnibus	One OS package; gitlab.rb service enable settings and roles select components.	Bundled PostgreSQL and Redis on the default node; PostgreSQL can be external.	external_url serves GitLab; Registry gets a port or URL, while Pages requires separate domain policy.	Roles can omit bundled services, but a reduced node then relies on the other nodes or external services. Roles, configuration, defaults.
Sentry self-hosted	One Compose stack; COMPOSE_PROFILES=errors-only or feature-complete selects one of two documented tiers, and Sentry reads that setting too.	The selected workers use the same PostgreSQL/PgBouncer, Valkey, Kafka, ClickHouse, and SeaweedFS services.	One nginx routes ingest to Relay and / to web.	errors-only omits feature-complete consumers; arbitrary per-feature selection and the data effects of changing tiers are not documented. Tier guide, Compose, nginx.
PostHog hobby	One Compose stack; no supported per-product switch is documented.	Its components use PostgreSQL, Redis/Valkey, Kafka, ClickHouse, and object storage.	Caddy fronts one domain and routes capture, flags, replay, and web paths.	A safe product subset is not documented. Hobby Compose, base Compose, self-hosting.
Supabase	One Compose distribution, with optional override services; the docs instruct operators to remove unused core service definitions and dependencies by hand.	One PostgreSQL instance serves services through distinct roles and schemas.	Envoy on port 8000 routes /auth/v1, /rest/v1, /storage/v1, and other API prefixes, with Studio at /.	A subset needs Compose editing; the docs do not say whether the fixed gateway routes also need changing. Docker guide, Compose, Envoy routes.
Wazuh	Server, indexer, and dashboard are separate installable components; Docker has single- and multi-node stacks.	The indexer holds the central event and alert data consumed by the other components.	Dashboard, server API, and indexer have separate ports, not a bundled single-origin gateway.	Components can be split across hosts, but the dashboard needs the server API and indexer; a server with no indexer at all is not established by these docs. Installation guide, architecture.
Elastic Stack	The example Compose fixes Elasticsearch nodes and Kibana; ECK’s eck-stack chart has component enabled flags.	Kibana connects to Elasticsearch; the reference stack has no SQL datastore.	Elasticsearch and Kibana expose separate ports; Kibana can be configured for a reverse-proxy base path.	Elasticsearch can run alone; Kibana requires an Elasticsearch endpoint. A Kibana-only ECK install against external Elasticsearch was not confirmed. Compose install, ECK values, Kibana settings.
Grafana	A single server image; optional panel, data-source, and app plugins install inside it.	SQLite by default, with MySQL/PostgreSQL configuration available; no other product shares that DB in the documented Docker install.	Serves port 3000 directly; root_url and serve_from_sub_path support a proxy subpath.	The server runs without plugins; an omitted data source is unavailable. Docker install, server configuration.
Authentik	Compose runs server and worker from one image plus PostgreSQL; optional outposts are managed separately.	Server and worker use one PostgreSQL database; the current Compose has no Redis service.	Server exposes 9000/9443; deployment can put a reverse proxy in front, and AUTHENTIK_WEB__PATH sets a subpath.	Server, worker, and PostgreSQL are the documented Compose base; outposts may be deployed separately. Compose, install, configuration.
Coder	One control-plane binary/container plus PostgreSQL in the reference Compose; its Helm chart does not install the database.	One PostgreSQL database, either built in for trials or configured externally.	CODER_ACCESS_URL names the control plane; optional wildcard subdomains serve apps.	The control plane can run alone with built-in DB; without a wildcard URL only path-based apps work. Serving the dashboard itself under a subpath was not confirmed. Compose, admin setup, wildcard URLs.
Mattermost	One application image plus PostgreSQL; Compose overlays choose bundled nginx or direct app publishing.	Its Compose uses a dedicated PostgreSQL service.	Bundled nginx fronts / on its domain, or the app exposes port 8065 without it.	Only the gateway choice is documented as an overlay; the base PostgreSQL remains. Subpath support was not confirmed. base Compose, nginx overlay, direct overlay.

GitLab’s service flags, Sentry’s tier, and the optional Supabase overrides show different ways to trim a bundle. None resolves Veltro’s particular requirement to keep identity and a path-based gateway running while selecting any combination of three independently released products.

Decision

Veltro is the brand and a thin shared layer: identity, the single-origin gateway, and the launcher. VectorFlow, CHAD, and Warden are the products. A customer installs any combination of them, including one product on its own. Core owns the public origin, identity, and shared infrastructure; installing a product adds its processes and dependencies without changing that origin. VELTRO_PRODUCTS=chad,warden denotes two installed products, not a new monolithic build.

1. Every product always runs behind Veltro identity. A “standalone” product is the core plus that one product, installed in one step. Products delete their local login, their own SAML and OIDC sign-in, and the VELTRO_INTEGRATED switch, so each product has exactly one authentication path: the gateway’s browser assertion (docs/adr/001-topology-and-session.md, nginx/veltro.conf).
2. Any subset installs, including none, and no product requires another. Warden without CHAD takes alerts from other detection platforms through its existing generic webhook sources (../warden/frontend/src/pages/WebhookSources.tsx).
3. The Veltro repository is the only install path: veltro install with a product selection for Compose, or the umbrella chart with per-product enabled values for Kubernetes, both through the ADR-013 CLI. Product repositories publish images and their Helm subcharts. Their Compose files serve product CI and development and are not a supported install.
4. Each product ships its own images and version pin. There is no combined image (rejected alternative 1).
5. With exactly one product installed, sign-in shows that product’s name and mark with “powered by Veltro” as secondary text, and / redirects to the product instead of showing the launcher. The fixed paths stay, so a VectorFlow-only install still serves /vectorflow from the suite image, and the plain VectorFlow image retires.
6. Product test suites cover authentication by minting signed assertions with their existing helpers (../vectorflow/src/server/services/suite/__tests__/browser-request-assertion.test.ts, ../chad/backend/tests/api/test_delegated_auth.py, ../warden/backend/tests/suite_envelope.py). Interactive checks run against the homelab. No product build ships a development-only sign-in stub.
7. A hosted offering, when there is one, runs one isolated installation per customer from the same published artifacts. It adds no shared deployment serving many customers and no separate hosted codebase (docs/adr/010-public-fixed-realm-product-boundary.md §8, docs/adr/011-vectorflow-cloud-disposition-ledger.md).

This changes ADR-013’s four-profile matrix once the migration is complete. Its CLI-first operation, independent product version pins, and Compose/Helm parity still apply (docs/adr/013-deployment-targets-and-operator-entry-point.md:13-27). This ADR specifies a target, not a shipped deployment; the current four profiles remain the contract until step 4 of the migration.

Focused milestone rules (2026-10-15)

VectorFlow-only and CHAD-only installs require core (identity and gateway) and their own infrastructure, with no sibling-product or Warden requirement. Product selection doesn’t make a release independent of core or its manifest.

Keep the current authority contracts: request assertions, PATs, grants, teams, workload exchange and approval policy. Home and settings aggregation stay. This milestone adds no mandatory cross-product responsibilities, identity extraction or alternative login path.

Warden is maintenance-only after its release removing fork_to_lake. Hold that published version with hold: warden=X.Y.Z on each suite cut unless a security, data-loss or existing-response correctness fix needs a new release. Cases, approvals, expiry/reversal workers and existing CHAD delivery and writeback continue. New SOAR features and vendor integrations aren’t prerequisites for the focused release.

Module contract

Every product module follows the same three rules when it runs inside the suite. They apply now, before any of the topology below ships, and they are what “installed in the suite” means for a module:

1. Identity owns sign-in and sign-out. A module never shows its own login page. A browser without a session goes to /login?next=<original path> at the site root, and logout ends the identity session (GET /api/auth/csrf, then POST /api/auth/logout, then /login).
2. The user is whoever identity names. On a browser request that’s the assertion’s sub; on a peer or MCP call made for someone it’s the v2 envelope’s actor_sub. The module finds its local user by that subject, or by email the first time, and links or creates it whatever credential that local row used before, because in suite mode the module’s own sign-in paths are unreachable. An email match that is already linked to a different subject is refused. An actor token without actor_sub (an older identity, or identity’s own status probe) still resolves by email. The name shown in the module comes from identity.
3. Access comes only from identity’s grants. The caller’s access is the grant in the assertion, or on an actor call the signed actor_app_role with actor_scope_kind and actor_team_id. A realm grant is the whole product; a team grant is that team. When the module needs other people’s access, such as assignable users, member pickers or team checks, it asks identity’s grant read (GET /api/authority/grants with its workload credential), which only ever answers for the module’s own product. The module keeps no second mapping of its own, and identity’s IdP group mappings grant a role across a whole product.

On 2026-09-26 CHAD met rules 2 and 3 but not 1; Warden met 3 but not 1 or 2 (it kept its local name); VectorFlow met none. Its resolver refused any local row whose authMethod wasn’t LOCAL, so an imported OIDC user was treated as signed out on every request. Each module’s fix is tracked against this section and lands in migration step 1. Once step 4 deletes each product’s local login, rule 1 holds because no other sign-in page exists.

Target topology

The table is the end state after datastore consolidation (migration step 5). Until then each installed product keeps its own PostgreSQL and Redis containers, as today. The base count assumes one replica per service except the existing two chad-worker replicas, bundled OpenSearch when CHAD is installed, no demo fixtures, and no optional TLS, monitoring, backup, agent, or cutover profile (compose/chad.yml:255-296,337-367, compose/{misp-mock,crowdsec-mock,monitoring,tls,backup}.yml).

Selection	Long-running services	One-shots	Containers
Core	veltro-nginx, veltro, suite-postgres, suite-redis	identity-migrate, identity-provision-signing-key, identity-provision-authority-fence, identity-provision-identity-workload	4 running + 4 one-shots = 8
Core + VectorFlow	Core + vectorflow-suite	Core + identity-provision-vectorflow-workload	5 running + 5 one-shots = 10
Core + CHAD	Core + chad-pgbouncer, chad-backend, two chad-worker, chad-frontend-suite, opensearch	Core + chad-migrate, identity-provision-chad-workload	10 running + 6 one-shots = 16
Core + Warden	Core + warden-backend, warden-worker, warden-frontend-suite	Core + identity-provision-warden-workload	7 running + 5 one-shots = 12
Core + all three	Union of all rows	Union of all rows	14 running + 8 one-shots = 22

The core’s PostgreSQL image is a pinned timescale/timescaledb:*-pg18 build (pg16 until the PostgreSQL 18 move, #1512) because VectorFlow needs TimescaleDB (compose/vectorflow.yml:62-82, ../vectorflow/docker/server/docker-compose.yml:26-44). It hosts separate databases veltro_identity, vectorflow, chad, and warden, with per-product roles and credentials, rather than a common schema or owner. The identity runtime/migration/import/recovery role separation must survive (compose/veltro.yml:41-92). The provisioning script creates only the installed product databases and roles; the identity database exists even with no product. The identity runtime and owner migration DSNs point to suite-postgres/veltro_identity. VectorFlow’s entrypoint builds DATABASE_URL from POSTGRES_HOST=suite-postgres, its product credential, and vectorflow; CHAD’s runtime uses chad-pgbouncer/chad, while chad-migrate goes directly to suite-postgres/chad; Warden’s POSTGRES_HOST=suite-postgres and POSTGRES_DB=warden produce its product DSN (compose/vectorflow.yml:5-15, compose/chad.yml:4-28,113-150, compose/warden.yml:4-15). No product receives another product’s database credential.

One persistent suite-redis replaces the four instances. Proposed URL mapping is VectorFlow redis://suite-redis:6379/0, CHAD cache /1, CHAD durable queue /2, and Warden /3, with product key prefixes as a second collision guard (compose/vectorflow.yml:10-15,84-96, compose/chad.yml:27-28,152-179, compose/warden.yml:11,86-95). Redis DB indices are namespaces, not failure or security boundaries. suite-redis needs AOF and a server-wide noeviction policy for the VF and CHAD queues. CHAD currently gives its cache allkeys-lru but its queue noeviction; one Redis process cannot retain both policies. Cache entries therefore need bounded TTL/cleanup, and queue backpressure, memory sizing, and recovery must be proved before consolidation (Redis eviction policies, compose/chad.yml:152-179). A single PostgreSQL or Redis failure will now affect every installed product, so backups, restore drills, capacity checks, and per-database restore procedures must cover the shared instances.

The .env selector is VELTRO_PRODUCTS, a comma-separated set drawn from vectorflow,chad,warden; an empty value means core only, and unknown or repeated names fail validation. The CLI derives core plus selected product --profile flags in stable order. VELTRO_INTEGRATED goes away, because every product container runs in suite mode (docker-compose.yml:3-22, cli/veltro_cli/lib/compose.py:56-102). VELTRO_PEER_*_URL and other peer URLs are supplied only when that peer is installed. Identity reports an absent peer as not_installed; VectorFlow’s version checks and cross-product connections apply only to installed products (compose/veltro.yml:20-33, compose/vectorflow.yml:49-58,166-172). Removing a product from this selector must not silently delete its data or identity grants.

Nginx retains every location and the single-origin assertion gate. It drops depends_on edges to product upstreams and uses its existing variable-based DNS resolution (docker-compose.yml:67-78, nginx/veltro.conf:40-75,83-167). An absent product’s upstream failure maps through error_page to an identity-served “not installed” page with a 503 status. Identity checks the installed set so an installed but unhealthy product stays a service-unavailable error, not a misleading installation message; API clients keep an error status. The launcher and AppSwitcher show only installed products (the shared registry’s buildSuiteApps already drops apps that aren’t listed, ../vectorflow/packages/shell/src/config.ts), and a single installed product follows Decision 5. Health and version probes do not interpret missing peers as outages. The /vectorflow prefix and its baked suite image remain mandatory (compose/vectorflow.yml:128-160). The demo mocks move to a named demo profile that veltro demo, the seed scripts, and scripts/reference_scenarios select explicitly; a normal up never starts them (compose/{misp-mock,crowdsec-mock}.yml).

Compose and Helm take the same product set and version pins through the CLI. Helm uses <product>.enabled booleans instead of the exact global.profile matrix; gateway and identity stay enabled. A selection yields the same paths, identity behavior, health contract, and upgrade order on both targets (docs/adr/013-deployment-targets-and-operator-entry-point.md:13-27,29-34, charts/veltro/templates/validate-profile.yaml:1-24, cli/veltro_cli/lib/k8s.py:12-45).

Migration is staged so each step ships through release-suite.yml before the next. Only the homelab runs these products today, so no step needs a deprecation window for outside users; the staging keeps every release gate green while the gates move.

1. Land the module-contract fixes in CHAD, Warden, and VectorFlow so each product meets the three rules in suite mode.
2. Composition. Add temporary additive suite-vectorflow, suite-chad, and suite-warden profiles alongside the old four, let core start on its own, and remove nginx’s product depends_on edges. Teach the CLI to read VELTRO_PRODUCTS, teach identity and the launcher to report not_installed, apply the single-product sign-in and redirect rule, and move the demo mocks to the demo profile. The old full path still selects all three products. Validate core-only and every subset.
3. Give Helm the per-product enabled selection while keeping the old profile render paths. Exercise identical product combinations, version pins, readiness, backup/restore, and upgrade behavior under the ADR-013 CLI contract.
4. Cutover. Remove full and the product-only standalone profiles from Veltro, rename the suite profiles to vectorflow, chad, and warden, and remove the Helm four-row validator. On 2026-09-26, 65 files outside build caches named the full profile: 33 in scripts/ (gate tests, verify and seed scripts, reference scenarios), 14 Compose files, 10 in the CLI, and the rest in workflows, charts, runbooks, and the README. Each one moves to a product selection in this step. In the product repositories, delete local login, product-owned SAML and OIDC, VELTRO_INTEGRATED, and the plain VectorFlow image, and mark their Compose files as CI and development fixtures.
5. Consolidate PostgreSQL and Redis into suite-postgres and suite-redis as its own change. Document and exercise dumps from each product database, role/database creation, restores, Redis queue preservation, verification, and rollback before retiring any volume, and prove CHAD’s bounded cache TTLs before the shared noeviction policy applies.

Consequences

• One product, two products, or all three install the same way, with one sign-in path per product and no fixed profile matrix. A core-only installation has one origin and identity without starting unused product services.
• Single-product installs get heavier. A VectorFlow-only install also runs the gateway, identity, identity’s database, and identity’s boot one-shots, which is what one authentication path per product costs.
• Product repositories stop supporting a deployment without Veltro. Running a product’s own Compose file outside CI is unsupported.
• Once step 5 lands, runtime and storage container counts fall from today’s 22 running plus six one-shots under full + opensearch to the proposed 14 plus six with all products. The target excludes both demo mocks and combines databases and Redis; it is not a claim about current deployments (docker-compose.yml:37-78, compose/{veltro,vectorflow,chad,warden,misp-mock,crowdsec-mock}.yml).
• Datastore consolidation reduces process count but expands the blast radius. Separate roles and databases limit accidental SQL access, while resource limits, backups, and an explicit Redis eviction policy become shared operational obligations.
• The CLI, identity status, launcher, nginx, Compose, Helm, and product-to-product assumptions must all learn the same installed-product set. Merely changing Compose profiles would leave the current hardcoded dependencies intact (cli/veltro_cli/lib/{compose,status,k8s}.py, compose/{veltro,vectorflow}.yml, docker-compose.yml:67-78).

Rejected alternatives

1. One omnibus image. A supervisor image containing identity and VectorFlow (Node), CHAD and Warden (Python), PostgreSQL, Redis, and OpenSearch would combine unrelated runtimes and storage lifecycles. VectorFlow’s /vectorflow base path is baked into a distinct suite image. Product releases have independent VF_VERSION, CHAD_VERSION, WARDEN_VERSION, and VELTRO_VERSION pins; one image would turn every pin update into a combined artifact without improving the Helm deployment target (compose/vectorflow.yml:128-160, versions.env:95,169,206,223, docs/adr/013-deployment-targets-and-operator-entry-point.md:13-27).
2. Keep all-or-one and trim containers. Dropping misp-mock and crowdsec-mock from full and merging PostgreSQL/Redis would reduce the default count, but profile == "full", nginx’s all-product dependencies, identity’s three peer URLs, and Helm’s exact matrix would still reject a CHAD-plus-Warden installation. The launcher would still assume that all three products exist (compose/{misp-mock,crowdsec-mock,veltro}.yml, docker-compose.yml:67-78, cli/veltro_cli/lib/compose.py:56-102, charts/veltro/templates/validate-profile.yaml:1-24).
3. Keep two sign-in modes per product. Each product keeps its own login standalone and uses Veltro assertions in the suite. That is today’s shape: dual-mode code in 27, 41, and 44 files, and a different contract failure in each product on 2026-09-26.
4. Standard OIDC in each product, with Veltro as one identity provider. Each product would keep its own login and map identity-provider claims to its own roles, which is the second mapping that Module contract rule 3 forbids. It would also replace the assertion contract in ADR-001 and ADR-008.
5. Install paths in every product repository. Four install paths drift. The product Compose defaults already lag the suite pins (see Context).
6. Warden requires CHAD. This turns away teams that already run another detection platform, and Warden’s generic webhook sources already serve them.
7. One shared hosted deployment for many customers. It reopens the scope ADR-010 defers and repeats the vectorflow-cloud history ADR-011 records.