Contributing detection rules and packs (Sigmatic)

Source of truth: docs/contributing/detection-rules.md (synced at build time — edit the original, not this copy). Detection content is Apache-2.0 (sigmatic/content/LICENSE): a rule you contribute can be copied and adapted without inheriting Sigmatic’s AGPL terms. The loader and API under sigmatic/backend/app/ stay AGPL — see sigmatic/LICENSING.md.

Where it goes

Path	Holds
content/rules/<product-or-family>/<slug>.yml	one Sigma rule per file
content/packs/<name>-<version>.yml	a sigmatic-detection-pack document

Every file starts with # SPDX-License-Identifier: Apache-2.0.

Before you write a rule, check SigmaHQ. Sigmatic subscribes to the SigmaHQ packages directly, so a rule that duplicates an upstream rule is better contributed to SigmaHQ — everyone gets it, including Sigmatic. This tree is for rules that are specific to a source Sigmatic integrates (the Pangolin/Traefik access family, the suite’s own audit planes) or that compose Sigmatic features SigmaHQ has no vocabulary for.

The rule shape Sigmatic executes

Sigmatic compiles Sigma to OpenSearch Lucene through pySigma (backend/app/services/sigma.py) and evaluates it in the shared DetectionPipeline across the push, worker and pull paths. What that means for an author:

• logsource names a family, not an index. Data sources are bound at deployment time; the family is what field mappings key on (POST /api/field-mappings/import, the ECS/OCSF preset families).
• Field names are Sigma field names. A deployment maps them onto its own documents. Do not hard-code a deployment’s field path.
• id is a UUID and is permanent. It is the identity deployments, exceptions and pack installations key on. A new detection is a new id, not an edited one.
• level is the rule’s confidence, and it seeds the risk score used by risk-based alerting. critical on a noisy pattern makes a deployment’s findings useless.
• IP matching. Sigma’s |cidr modifier needs an OpenSearch ip-typed field; dynamically-mapped log fields are text/keyword. Prefer exact or |startswith matching unless a data source explicitly types the field.
• False-positive notes are part of the rule. falsepositives: is what an operator reads before deploying.

Validating before you push

cd sigmatic
pip install -r backend/requirements.txt
python3 scripts/validate_content.py       # the gate CI runs
python3 scripts/check_spdx.py

scripts/validate_content.py runs the product’s own validators, so a green run means Sigmatic can load your content:

Check	Runs
Sigma syntax and schema	app.services.sigma.sigma_service.validate_rule — the same parser as the rule editor and the deployment gate
Pack integrity	app.services.detection_pack.verify_integrity — parse, SHA-256 over the canonical bytes, and every embedded rule
Naming and licence	pack filed as <name>-<version>.yml; SPDX header in the first five lines

.github/workflows/content-validation.yml runs exactly that on your pull request, on the existing shared self-hosted X64 pool.

Field validation and false positives against real data

The gate above is static. To see whether a rule’s fields exist and how noisy it is on a real deployment, use the detection-as-code CI API against a data source of the same family:

POST /api/rule-ci/check          # rule YAML + data source
POST /api/rule-ci/{rule_id}/check

It returns lint, field_validation, fp_backtest and coverage checks (backend/app/services/rule_ci.py). The OpenSearch-dependent checks report skipped when no cluster is configured, so a passed report with a skipped backtest is not a claim about noise.

Building a pack

A pack is one deterministic YAML document carrying its rules, its provenance and a SHA-256 over its own canonical bytes. It cannot be edited in place — rebuild it:

python3 - <<'PY'
from app.services.detection_pack import PackRule, export_pack

rule = open("content/rules/linux/ssh-brute-force.yml").read()
print(export_pack(
    [PackRule(rule_id="linux-auth-ssh-brute-force",
              title="SSH brute force",
              yaml=rule)],
    name="linux-auth", version="1.0.0",
    description="Linux authentication detections.",
    source="first-party",
))
PY

Prepend the SPDX comment to the emitted document; the checksum covers the parsed document, so a comment does not affect it. version is semantic (X.Y.Z), provenance.source records where the content came from, and compatibility.sigmatic records the version it was built against — apply refuses a pack the deployment does not satisfy.

Review bar

A reviewer will look for: a permanent UUID id; a logsource family Sigmatic can map; a detection that survives rule-ci field validation; a level that matches the pattern’s confidence; falsepositives where the pattern is known to fire on benign activity; and tags with the ATT&CK technique (attack.tXXXX) so the coverage matrix and pack simulation can count it.

Correlations are Sigma correlation rules (correlation.type of event_count, value_count, temporal, temporal_ordered) and follow the same review bar, plus: a timespan an operator can reason about, and group-by fields that exist in the base rules’ mappings.